Re: Contributor grants request

[Andrey Gura]

Hi, Rodion!

Could you please provide your Apache JIRA login name?

On Mon, Apr 5, 2021 at 5:37 PM Rodion Smolnikov  wrote:
>
> Hello!
> Can I get contributor access?
> Task https://issues.apache.org/jira/browse/IGNITE-14474
>
> --
>
> Rodion Smolnikov
>
> +7(915)256-54-81
> Software Developer, Russia
>
> *gridgain.com *

Re: Contributor grants request

[Andrey Gura]

Granted Contributor role. Feel free to assign issues.

On Tue, Apr 6, 2021 at 6:25 PM Smolnikov  wrote:
>
> My username is "Smolnikov"
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Metrics naming formalization.

[Andrey Gura]

Hi, Mikhail!

I've looked at the problem and the fix. The root cause of the problem
is MetricsRegistry design which allows to register metrics in two
ways:

- using a family of methods like MetricRegistre.register(..,
XxxSupplier, ...) and MetricRegistry.xxxMetric. These methods firstly
create metrics and take into account registry's name. These methods
provide the right way for registering metrics in MetricRegistry.
- using the method MetricRegistry.register(Metric metric). This method
takes as a parameter an already created metric which was created out
of the metric registry context. This method is the wrong (roughly
speaking) way for registering metrics in MetricRegistry.

Using the right approach for metrics registering is a guaranteed way
to avoid problems like described in [1]. Moreover, this approach does
not require any metrics naming formalization because it hides internal
naming details from an user.

So, IMHO, the issue [2] ideally should be re-fixed in a right way.

Also, ideally, we should limit the visibility scope of  metric classes
constructors and do not allow instantiate these classes directly.

[1] https://issues.apache.org/jira/browse/IGNITE-14376
[2] https://issues.apache.org/jira/browse/IGNITE-14428

On Fri, Apr 9, 2021 at 1:25 PM Mikhail Petrov  wrote:
>
> Hello, Igniters.
>
> I faced the following problem with the metrics - JmxMetricExporter fails
> to export discovery metrics (ticket - [1]).
>
> The main reason: JMX exporter assumes that each metric name must start
> with the name of the registry it belongs to, but discovery metrics do
> not obey this naming convection.
>
> I propose to formalize the names of the metrics included in the metrics
> registry so that they always start with the name of the register. And
> add corresponding assertions (ticket - [2]).
> It helps to make the metric naming consistent and fixes the mentioned
> above issue.
>
> The following metric names are proposed to be changed:
>
> JoinedNodes -> io.discovery.JoinedNodes
> LeftNodes -> io.discovery.LeftNodes
> FailedNodes -> io.discovery.FailedNodes
> PendingMessagesRegistered -> io.discovery.PendingMessagesRegistered
> CommunicationErrors -> io.discovery.CommunicationErrors
> CurrentTopologyVersion -> io.discovery.CurrentTopologyVersion
>
> Any objections?
>
>
> [1] - https://issues.apache.org/jira/browse/IGNITE-14376
> [2] - https://issues.apache.org/jira/browse/IGNITE-14428
>

Re: Exceeding the DataStorageConfiguration#getMaxWalArchiveSize due to historical rebalance

[Andrey Gura]

Stan

> If archive size is less than min or more than max then the system 
> functionality can degrade (e.g. historical rebalance may not work as 
> expected).

Why does the condition "archive size is less than min" lead to system
degradation? Actually, the described case is a normal situation for
brand new clusters.

I'm okay with the proposed minWalArchiveSize property. Looks like
relatively understandable property.

On Sun, May 9, 2021 at 7:12 PM Stanislav Lukyanov
 wrote:
>
> Discuss this with Kirill verbally.
>
> Kirill showed me that having the min threshold doesn't quite work.
> It doesn't work because we no longer know how much WAL we should remove if we 
> reach getMaxWalArchiveSize.
>
> For example, say we have minWalArchiveTimespan=2 hours and 
> maxWalArchiveSize=2GB.
> Say, under normal load on stable topology 2 hours of WAL use 1 GB of space.
> Now, say we're doing historical rebalance and reserve the WAL archive.
> The WAL archive starts growing and soon it occupies 2 GB.
> Now what?
> We're supposed to give up WAL reservations and start agressively removing WAL 
> archive.
> But it is not clear when can we stop removing WAL archive - since last 2 
> hours of WAL are larger than our maxWalArchiveSize
> there is no meaningful point the system can use as a "minimum" WAL size.
>
> I understand the description above is a bit messy but I believe that whoever 
> is interested in this will understand it
> after drawing this on paper.
>
>
> I'm giving up on my latest suggestion about time-based minimum. Let's keep it 
> simple.
>
> I suggest the minWalArchiveSize and maxWalArchvieSize properties as the 
> solution,
> with the behavior as initially described by Kirill.
>
> Stan
>
>
> > On 7 May 2021, at 15:09, ткаленко кирилл  wrote:
> >
> > Stas hello!
> >
> > I didn't quite get your last idea.
> > What will we do if we reach getMaxWalArchiveSize? Shall we not delete the 
> > segment until minWalArchiveTimespan?
> >
> > 06.05.2021, 20:00, "Stanislav Lukyanov" :
> >> An interesting suggestion I heard today.
> >>
> >> The minWalArchiveSize property might actually be minWalArchiveTimespan - 
> >> i.e. be a number of seconds instead of a number of bytes!
> >>
> >> I think this makes perfect sense from the user point of view.
> >> "I want to have WAL archive for at least N hours but I have a limit of M 
> >> gigabytes to store it".
> >>
> >> Do we have checkpoint timestamp stored anywhere? (cp start markers?)
> >> Perhaps we can actually implement this?
> >>
> >> Thanks,
> >> Stan
> >>
> >>>  On 6 May 2021, at 14:13, Stanislav Lukyanov  
> >>> wrote:
> >>>
> >>>  +1 to cancel WAL reservation on reaching getMaxWalArchiveSize
> >>>  +1 to add a public property to replace 
> >>> IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE
> >>>
> >>>  I don't like the name getWalArchiveSize - I think it's a bit confusing 
> >>> (is it the current size? the minimal size? the target size?)
> >>>  I suggest to name the property geMintWalArchiveSize. I think that this 
> >>> is exactly what it is - the minimal size of the archive that we want to 
> >>> have.
> >>>  The archive size at all times should be between min and max.
> >>>  If archive size is less than min or more than max then the system 
> >>> functionality can degrade (e.g. historical rebalance may not work as 
> >>> expected).
> >>>  I think these rules are intuitively understood from the "min" and "max" 
> >>> names.
> >>>
> >>>  Ilya's suggestion about throttling is great although I'd do this in a 
> >>> different ticket.
> >>>
> >>>  Thanks,
> >>>  Stan
> >>>
>   On 5 May 2021, at 19:25, Maxim Muzafarov  wrote:
> 
>   Hello, Kirill
> 
>   +1 for this change, however, there are too many configuration settings
>   that exist for the user to configure Ignite cluster. It is better to
>   keep the options that we already have and fix the behaviour of the
>   rebalance process as you suggested.
> 
>   On Tue, 4 May 2021 at 19:01, ткаленко кирилл  
>  wrote:
> >  Hi Ilya!
> >
> >  Then we can greatly reduce the user load on the cluster until the 
> > rebalance is over. Which can be critical for the user.
> >
> >  04.05.2021, 18:43, "Ilya Kasnacheev" :
> >>  Hello!
> >>
> >>  Maybe we can have a mechanic here similar (or equal) to checkpoint 
> >> based
> >>  write throttling?
> >>
> >>  So we will be throttling for both checkpoint page buffer and WAL 
> >> limit.
> >>
> >>  Regards,
> >>  --
> >>  Ilya Kasnacheev
> >>
> >>  вт, 4 мая 2021 г. в 11:29, ткаленко кирилл :
> >>
> >>>  Hello everybody!
> >>>
> >>>  At the moment, if there are partitions for the rebalance for which 
> >>> the
> >>>  historical rebalance will be used, then we reserve segments in the 
> >>> WAL
> >>>  archive (we do not allow cleaning the WAL archive) until the 
> >>> rebalance for
> >>>  all cache groups is over.
> >>>
> >>>

Re: Apache Ignite 2.11

[Andrey Gura]

Maksim,

why do you want to include just one phase of functionality into
release instead of full implementation? History of Ignie development
shows (at least from my point of view) that it is bad practice. As
result users will have unfinished and badly tested functionality. We
have (or had in the past) some examples of such things: MVCC, IEP-35,
etc.

On Mon, May 17, 2021 at 5:11 PM Алексей Гидаспов  wrote:
>
> Hi, Dmitriy!
>
> I suggest this timeline
>
> Scope Freeze: May 31, 2021
> Code Freeze: 14 June, 2021
> Voting Date: 21 June, 2021
> Release Date: 28 June, 2021
>
> пт, 14 мая 2021 г. в 11:32, Maksim Timonin :
>
> > Hi, I'd like to include Phase 1 of index queries feature [1] to the next AI
> > release. Currently there are 2 tickets in progress ([2], [3]), and they
> > will be ready for review next week. What is a time of code freeze if it's
> > going to release AI 2.11 at the end of June?
> >
> > [1]
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-71%3A+Public+API+for+secondary+index+search
> > [2] https://issues.apache.org/jira/browse/IGNITE-14699
> > [3] https://issues.apache.org/jira/browse/IGNITE-14703
> >
> > On Thu, May 13, 2021 at 5:27 PM Dmitry Pavlov  wrote:
> >
> > > Hi Alexey,
> > >
> > > :pmc to sign distribution:
> > > Formally, only PMC member can be a release manager, so part of activities
> > > should be picked up by committer and PMC member. If noone else want to
> > help
> > > you here, I would.
> > >
> > > :timeline:
> > > And, could you estimate all phases of the release.
> > >
> > > :scope:
> > > I guess we can just pick up master current state and release it. But if
> > > someone has some ideas if we should wait for particular feature to be
> > > completed before branch divergence/release candidate build, please let
> > know.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > On 2021/05/13 09:45:02, Алексей Гидаспов  wrote:
> > > > Dear Ignite Community!
> > > >
> > > > I suggest starting Apache Ignite 2.11 release activities. Also suggest
> > > > myself to be a release manager for this release. I plan release at the
> > > end
> > > > of june 2021.
> > > >
> > > > ___
> > > >
> > > > Best Regards,
> > > > Alexey Gidaspov
> > > >
> > >
> >

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Andrey Gura]

Hi,

I think that scope should be limited by public API  for beginning.
Also I don't sure that we should support specific tags like @apiNote,
@implSpec, @implNote.

Let's move using the following plan:

1. Create an empty (effectively) checkstyle config for javadoc
checking and commit it to the repository. After this step it will be
possible to configure TC in order to use this configuration.
2. Configure TC.
3. Commit a non-empty checkstyle config, but all modules should be
excluded from checking on this step.
4. For each module create a JIRA issue. Module maintainers fix
corresponding tickets and remove module exclusion from checking.
5. Add information about javadoc validation targets to DEVNOTES.md
file (could be done on step 3)

Any objections?

On Tue, Apr 20, 2021 at 11:40 AM Andrey Mashenkov
 wrote:
>
> I've fixed the PR.
>
> Now,
> 1. Javadoc style is only checked if it is present for the element. All
> declared javadoc tags MUST have a description.
> So, one should either write correct javadoc or don't write at all.
> 2. Additional checks performed for non-internal and non-test classes. All
> public and protected elements must have non-emtpy javadoc.
>
> So, checkstyle detects 500+ missed descriptions (missed javadocs, tags
> descriptions, tags themselves) in public scope
> and 180+ bad-styles (missed brased, spaces, empty-lines) javadocs in whole
> codebase.
>
> I'd suggest to force non-empty javadocs for all non-test classes including
> internal (but their members) as well.
>
>
>
> On Mon, Apr 19, 2021 at 6:37 PM Andrey Mashenkov 
> wrote:
>
> > Alexey,
> >
> > These are bad examples and unfortunately, most of the Ignite code doesn't
> > look self-descriptive.
> > (Do you feel the difference between @SerializableTransient and
> > @TransientSerializable?)
> >
> > To understand the semantic of e.g. 'metricsHistSize' you have to analyze
> > its usages.
> > Getter and setter for this property look better, but it still not clear
> > what happens with metric values above the limit?
> >
> > I have another example, one of the core components GridDhtPartitionDemander
> > has totally useless javadoc.
> > It is obviously has nothing with thread pool + "local cache" phrase looks
> > very confusing.
> >
> > /**
> >  * Thread pool for requesting partitions from other nodes and populating 
> > local cache.
> >  */
> > public class GridDhtPartitionDemander
> >
> > To understand the purpose of this internal component I have to analyze its
> > code and usages.
> > However, if one will face unexpected behavior, he/she may be confused if
> > it is a lack of javadoc or wrong behavior.
> >
> > There is no way to restrict or avoid such issues with any checks.
> > Agree, meaningful javadocs as self-descriptive code is more about culture
> > and discipline.
> >
> > The absence of javadoc on internal API, unreasonably raises the entry
> > threshold for new developers and makes the development of intra-component
> > interaction harder.
> > I admit a high-level description for public classes may be enough to
> > resolve this without pushing developers to write empty/useless docs for
> > every method/field.
> >
> >
> > On Mon, Apr 19, 2021 at 5:21 PM Alexey Kukushkin <
> > kukushkinale...@gmail.com> wrote:
> >
> >> I personally do not understand why we need mandatory javadoc even in
> >> public
> >> modules. I think javadoc is needed only when the code is not
> >> self-descriptive. What value does a javadoc like this bring
> >> <
> >> https://github.com/apache/ignite/blob/2.10.0/modules/core/src/main/java/org/apache/ignite/configuration/IgniteConfiguration.java
> >> >
> >> to a developer:
> >>
> >> /** Default metrics history size (value is {@code 1}). */
> >> public static final int DFLT_METRICS_HISTORY_SIZE = 1;
> >>
> >> /** Metrics history time. */
> >> private int metricsHistSize = DFLT_METRICS_HISTORY_SIZE;
> >>
> >> BTW, I picked a random example and it already has a copy-paste mistake in
> >> the javadoc, which is there for years. I think that indicates it has no
> >> real value and developers just do it to comply with the rule.
> >>
> >> Some say mandatory javadoc is for the discipline but I think Apache Ignite
> >> developers are mature enough to understand when additional documentation
> >> is
> >> really required.
> >>
> >>
> >>
> >> пн, 19 апр. 2021 г. в 16:37, Ivan Pavlukhin :
> >>
> >> > Hi Andrey and Igniters,
> >> >
> >> > Sorry if I misread something but I have totally different opinion in
> >> > one aspect. In my mind it is much better in practice to follow strict
> >> > rules for public API javadocs but not for internals. I would use
> >> > static checks for API packages and disable them for internal ones and
> >> > refine code readability during code review. Main motivation here is
> >> > that ubiquitous javadocs did not work well in ignite-2 and I believe
> >> > it would not in ignite-3.
> >> >
> >> > 2021-04-19 13:30 GMT+03:00, Andrey Mashenkov <
> >> andrey.mashen...@gmail.com>:
> >> > > Hi 

Re: [DISCUSSION] Code style. Variable abbrevations

[Andrey Gura]

Hi,

I understand that this rule seems too strict or may be weird. But
removing the rule could lead to review comments like "use future
instead of fut". So my proposal is to change rule from "required" to
"recommended".

On Wed, Jun 16, 2021 at 2:49 PM Valentin Kulichenko
 wrote:
>
> Konstantin, thanks for chiming in.
>
> That's exactly my concern. Overformalization is generally not a good thing.
> Someone used "mess" to abbreviate "message"? That is surely not a good
> coding style, but that's what code reviews are for. I believe that our
> committers are more than capable of identifying such issues.
>
> At the same time, with the current rules, we are *forced* to use
> abbreviations like "locAddrGrpMgr", whether we like it or not. All it does
> is makes it harder to contribute to Ignite, without providing any clear
> value.
>
> -Val
>
> On Thu, Jun 10, 2021 at 9:46 AM Konstantin Orlov 
> wrote:
>
> > +1 for making this optional
> >
> > Common abbreviation rules is good for sure, but sometimes I getting sick
> > of all those locAddrGrpMgr.
> >
> >
> > --
> > Regards,
> > Konstantin Orlov
> >
> >
> >
> >
> > > On 7 Jun 2021, at 14:31, Nikolay Izhikov  wrote:
> > >
> > > Hello, Anton, Alexei.
> > >
> > > Thanks for the feedback.
> > >
> > > Personally, I’m pretty happy current abbreviation rules too.
> > > Let see what we can do to make our codebase even more consistent.
> > >
> > >> 7 июня 2021 г., в 13:23, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com> написал(а):
> > >>
> > >> -1
> > >> Common abbrevs add quality to the code.
> > >>
> > >> пн, 7 июн. 2021 г. в 12:38, Anton Vinogradov :
> > >>
> > >>> -1 here.
> > >>> We can fix the code and set up the rule.
> > >>>
> > >>> This will help to prevent having a weird abbreviation like "mess" (from
> > >>> "message") or "ign" (from "Ignite").
> > >>> Also, the abbreviations list (hardcoded at IDEA plugin) allows to have
> > same
> > >>> names across the whole code, this should simplify the reading.
> > >>>
> > >>> On Sat, Jun 5, 2021 at 10:49 PM Valentin Kulichenko <
> > >>> valentin.kuliche...@gmail.com> wrote:
> > >>>
> >  I also support removing this requirement. It’s not the first time
> > someone
> >  brings this up, and so far we haven’t been able to fix it. Not worth
> > it
> > >>> in
> >  my view.
> > 
> >  -Val
> > 
> >  On Sat, Jun 5, 2021 at 11:54 AM Nikolay Izhikov 
> >  wrote:
> > 
> > > Hello, guys.
> > >
> > > Thanks for the feedback.
> > >
> > > Dmitry,
> > >
> > >> Manual rule enforcement saves a couple of symbols in code
> > >
> > > I’m talking about automatic check.
> > > We can implement it easily.
> > > So, if we decide to keep this rule all we need is to fix current
> > > violations (several thousand!).
> > > After it, checkstyle will automatically enforce this rule.
> > > As you may know, currently, any PR checked according to our
> > checkstyle
> > > rules.
> > > Please, take a look at little green check sign after PR name.
> > >
> > >
> > >> 5 июня 2021 г., в 00:57, Dmitry Pavlov 
> >  написал(а):
> > >>
> > >> +1 for removal. Cnt and count both seem to be fine.
> > >>
> > >> Manual rule enforcement saves a couple of symbols in code, but
> > >>> requires
> > > some time from both contributor and reviewer.
> > >>
> > >> Sincerely,
> > >> Dmitriy Pavlov
> > >>
> > >> On 2021/06/04 19:18:37, Pavel Tupitsyn 
> > wrote:
> > >>> In my opinion, we should remove this rule.
> > >>> Looks like a waste of time. freq or frequency, cnt or count, it is
> >  fine
> > >>> either way.
> > >>>
> > >>> On Fri, Jun 4, 2021 at 7:39 PM Nikolay Izhikov <
> > nizhi...@apache.org
> > 
> > > wrote:
> > >>>
> >  Hello, Igniters.
> > 
> >  Right now, we have the rule to use some predefined list of
> >  abbrevation
> > > for
> >  variable names [1].
> >  Some of the reviewers ask to follow this rule strictly.
> > 
> > > It is required to use abbreviated form for code consistency.
> > 
> >  I tried to implement this rule in form of checkstyle check [2] and
> > >>> it
> >  seems like many of use doesn’t follow this requirement.
> >  My check found 4124 violation in core module.
> > 
> >  Should we make this rule optional in documentation or should we
> >  remove
> > > it
> >  completely?
> >  Or should someone proceed and fix all the violations?
> > 
> >  WDYT?
> > 
> > 
> >  Example of output:
> >  ```
> >  [ERROR]
> > 
> > >
> > 
> > >>>
> > /Users/sbt-izhikov-nv/work/ignite/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/IgnitePdsWithTtlTest.java:94:
> >  Abbrevation should be used for CACHE_NAME_LOCAL_ATOMIC! Please,

Re: [DISCUSSION] Add cache statistics switch to control script

[Andrey Gura]

Hi, Ilya

Could you please provide more details about how it should work (in
issue description)? E.g. what if I pass the cache group name instead
of the name of a particular cache? What if I will not pass any cache
names?

Also we don't use the word "statistics" anymore (while it is still
called statistics in configuration). What about the word "metrics"?

On Wed, Jun 16, 2021 at 2:31 PM Shishkov Ilya  wrote:
>
> Dear Ignite Community!
> I propose to discuss the addition of a new feature for control script:
> cache statistics switch (enabling / disabling). Currently, cache statistics
> can be toggled only via IgniteVisorCmd or JMX.
> I've created the corresponding issue [1] and suggest adding an extra option
> to a '--cache' command of the control script, eg.:
>
> --cache statistics enable|disable [cacheName1,...,cacheNameN]
>
> Please, tell me if you have any suggestions or remarks against this feature.
>
> 1. https://issues.apache.org/jira/browse/IGNITE-14913

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Andrey Gura]

Hi,

Unfortunately, such things as the "many private APIs" rule can't be
formalized for any validation tool. So we have to choose between rules
like "everything should be documented" and "private API documentation
is not mandatory".

The community prefers the second approach. I don't want to argue about
it (although I prefer to document the private API ). My goal is
getting a consistent and customizable way to validate javadoc where it
is mandatory (public API). It also could be applied to private API,
but it should be another rule set which doesn't require javadoc but
validates existing javadoc.

On Wed, Jun 16, 2021 at 11:32 PM Nikita Ivanov  wrote:
>
> Hi,
> In my strong opinion Javadoc is a code. Any errors in Javadoc are
> equal to the bug in the code and must be treated as such. Proper and
> extensive Javadoc for all public and many private APIs is absolutely
> essential for this project, its growth and maturity.
>
> I'm surprised this community still needs to debate fundamental
> engineering issues like that...
> --
> Nikita Ivanov
>
> On Wed, Jun 16, 2021 at 4:47 AM Andrey Gura  wrote:
> >
> > Hi,
> >
> > I think that scope should be limited by public API  for beginning.
> > Also I don't sure that we should support specific tags like @apiNote,
> > @implSpec, @implNote.
> >
> > Let's move using the following plan:
> >
> > 1. Create an empty (effectively) checkstyle config for javadoc
> > checking and commit it to the repository. After this step it will be
> > possible to configure TC in order to use this configuration.
> > 2. Configure TC.
> > 3. Commit a non-empty checkstyle config, but all modules should be
> > excluded from checking on this step.
> > 4. For each module create a JIRA issue. Module maintainers fix
> > corresponding tickets and remove module exclusion from checking.
> > 5. Add information about javadoc validation targets to DEVNOTES.md
> > file (could be done on step 3)
> >
> > Any objections?
> >
> > On Tue, Apr 20, 2021 at 11:40 AM Andrey Mashenkov
> >  wrote:
> > >
> > > I've fixed the PR.
> > >
> > > Now,
> > > 1. Javadoc style is only checked if it is present for the element. All
> > > declared javadoc tags MUST have a description.
> > > So, one should either write correct javadoc or don't write at all.
> > > 2. Additional checks performed for non-internal and non-test classes. All
> > > public and protected elements must have non-emtpy javadoc.
> > >
> > > So, checkstyle detects 500+ missed descriptions (missed javadocs, tags
> > > descriptions, tags themselves) in public scope
> > > and 180+ bad-styles (missed brased, spaces, empty-lines) javadocs in whole
> > > codebase.
> > >
> > > I'd suggest to force non-empty javadocs for all non-test classes including
> > > internal (but their members) as well.
> > >
> > >
> > >
> > > On Mon, Apr 19, 2021 at 6:37 PM Andrey Mashenkov 
> > > 
> > > wrote:
> > >
> > > > Alexey,
> > > >
> > > > These are bad examples and unfortunately, most of the Ignite code 
> > > > doesn't
> > > > look self-descriptive.
> > > > (Do you feel the difference between @SerializableTransient and
> > > > @TransientSerializable?)
> > > >
> > > > To understand the semantic of e.g. 'metricsHistSize' you have to analyze
> > > > its usages.
> > > > Getter and setter for this property look better, but it still not clear
> > > > what happens with metric values above the limit?
> > > >
> > > > I have another example, one of the core components 
> > > > GridDhtPartitionDemander
> > > > has totally useless javadoc.
> > > > It is obviously has nothing with thread pool + "local cache" phrase 
> > > > looks
> > > > very confusing.
> > > >
> > > > /**
> > > >  * Thread pool for requesting partitions from other nodes and 
> > > > populating local cache.
> > > >  */
> > > > public class GridDhtPartitionDemander
> > > >
> > > > To understand the purpose of this internal component I have to analyze 
> > > > its
> > > > code and usages.
> > > > However, if one will face unexpected behavior, he/she may be confused if
> > > > it is a lack of javadoc or wrong behavior.
> > > >
> > > > There is no way t

Re: IEP-61 Transaction API desing for Ignite 3

[Andrey Gura]

Alexey,

could you please describe Transaction interface?

Also it would be great to have a couple examples of using the proposed API.

On Tue, Jul 13, 2021 at 4:43 PM Alexei Scherbakov
 wrote:
>
> Folks,
>
> I've prepared a PR implementing my vision of public transactions API.
>
> API is very simple and similar to Ignite 2, but has some differences.
>
> More details can be found here [1]
>
> Share your thoughts.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15086
>
> --
>
> Best regards,
> Alexei Scherbakov

Re: [Discussion] Race on heartbeat update in checkpoint thread

[Andrey Gura]

Alexey, Ilya,

I took a look at the problem and corresponding fixes. It seems that
you are both right. But Alexey's fix looks like some kind of hack to
me.

We have two problems:

1. Heartbeat update from thread that will complete a future.

I agree with Ilya and Andrey. Only a critical worker itself can update
heartbeat. So, removal of `res.listen(fut ->
heartbeatUpdater.updateHeartbeat());` from
CheckpointContextImpl#executor() method is a good idea. It will avoid
a race and makes Alexey's fix redundant.

2. blockingSection in checkpointer thread really can never detect the
lack of pending tasks progress.

Alexey is right. It is a problem. But what I actually see:
checkpointWritePageThreads and checkpointCollectInfoThreads which
execute pending tasks, are actually critical themselves and should be
also monitored by WorkersRegistry. In such a case the checkpointer
thread can safely use blocking sections at places designated by Ilya
and forementioned threads should also use blocking sections on IO
operations.

So proposal:

- Remove Alexey's fix
- Remove asynchronous heartbeat update
- Add blocking sections (fix of Ilya)
- Monitor checkpointWritePageThreads and checkpointCollectInfoThreads
if they exist (it depends on configuration)

WDYT?

On Thu, Jul 29, 2021 at 5:48 AM Ilya Kazakov  wrote:
>
> Guys. I have discussed just these changes. Look at my PR, please. What do
> you think?
>
> https://issues.apache.org/jira/browse/IGNITE-15192
> https://github.com/apache/ignite/pull/9280/files
>
> чт, 22 июл. 2021 г. в 22:30, Andrey Mashenkov :
>
> > Hi guys,
> >
> > I think updateHeartBeat() method was misused in the future listener and
> > this must be fixed.
> >
> > Actually, GridWorker.heartbeatTs Javadoc says that field is updated by the
> > worker itself.
> > It is consistent with WorkProgressDispatcher.updateHearbeat() javadoc,
> > which said "Notifying dispatcher that work is in progress and thread didn't
> > freeze."
> > GridWorker.heartbeatTs field is marked volatile just to provide consistent
> > visibility guarantee to a watcher.
> >
> > So, CheckpointPagesWriter violates this contract, when runs in another
> > thread(s).
> > But it works fine just because the main (Checkpointer) thread waits for
> > all the Writers to finish their work before it can fall into the blocking
> > section, therefore there is no race possible.
> >
> > As for the broken case, there are 2 possible solutions,
> > 1. Main thread must wait for all children's tasks to finish before going
> > into the blocking section.
> > 2. Or make updateHeartbeat consistent with the blockingSectionBegin/End.
> > Seems, there is no need to mark the method as synchronized, but use call
> > AtomicLongFieldUpdater.getAndUpdate(this, v -> v == Long.MAX_VALUE ? v :
> > U.currentTimeMillis()).
> >
> > I'd suggest
> > * Remove the "thread" mentioning from WorkProgressDispatcher interface to
> > avoid confusion with the current usage and make WorkProgressDispatcher more
> > general.
> > * Avoid unsafe heartbeatUpdater leak to the outside via wrapping
> > CheckpointContextImpl.heartbeatUpdater instance to perform consistent
> > heartbeat update from p.2 above.
> >
> > Does it make sense?
> >
> > On Wed, Jul 21, 2021 at 11:12 AM Alex Plehanov 
> > wrote:
> >
> >> Ilya,
> >>
> >> Race only affects the future listener (which only updates heartbeat), but
> >> not checkpoint listeners, so no other bugs possible caused by this race
> >> except wrong heartbeat update.
> >>
> >> As I've written before, I think it's not a good idea to wait for other
> >> threads by the critical thread in the blocking section. If these threads
> >> are not monitored we can never detect lack of progress and never trigger
> >> the failure handler. Perhaps a good solution will be to register async
> >> threads as critical threads (additionally to waiting for these threads in
> >> the blocing section) and update their own heartbeat. But the current fix
> >> is
> >> required too, to avoid such problems for other critical threads in the
> >> future.
> >>
> >> ср, 21 июл. 2021 г. в 06:29, Ilya Kazakov :
> >>
> >> > 1.
> >> > I mean calling listeners in CheckpointWorkflow.markCheckpointBegin():
> >> >
> >> > This
> >> > --
> >> > for (CheckpointListener lsnr : dbLsnrs)
> >> > lsnr.beforeCheckpointBegin(ctx0);
> >> >
> >> > ctx0.awaitPendingTasksFinished();
> >> > --
> >> >
> >> > And this:
> >> >
> >> > --
> >> > // Listeners must be invoked before we write checkpoint record to WAL.
> >> > for (CheckpointListener lsnr : dbLsnrs)
> >> > lsnr.onMarkCheckpointBegin(ctx0);
> >> >
> >> > ctx0.awaitPendingTasksFinished();
> >> > --
> >> >
> >> > Inside lsnr.beforeCheckpointBegin(ctx0) and
> >> > lsnr.onMarkCheckpointBegin(ctx0) we call
> >> CheckpointContextImpl.executor()
> >> > which submit heartbeat update tasks in threadpool.

Re: [DISCUSSION] Code style for Ignite 3

[Andrey Gura]

Looks good to me. But Idea configuration for style check is not
enough. It helps developers but does not automate style checking.

Checkstyle project provides ready to use config based on Google Code
Style [1]. I hope it matches well with Idea config and we'll avoid any
confusing incidents.

Let's start with this convention and adopt it to our needs if needed.

[1] 
https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml

On Fri, Aug 20, 2021 at 12:02 PM Alexei Scherbakov
 wrote:
>
> +1
>
> пт, 20 авг. 2021 г. в 10:54, Alexander Polovtcev :
>
> > Hi, Val. This is an extremely welcome change, thank you!
> >
> > On Fri, Aug 20, 2021 at 12:17 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > I would like to discuss a potential change to the coding guidelines for
> > > Ignite 3. Currently, we're using the existing guidelines inherited from
> > > Ignite 2, which are described here:
> > > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
> > >
> > > Current guidelines, however, exist for many years and have several
> > issues.
> > > They are cumbersome, carry a lot of legacy stuff, and can't be automated.
> > > Every now and then, they seem to cause questions and confusion.
> > >
> > > While it's hard to make drastic changes in Ignite 2, we still have a
> > great
> > > opportunity to update the guidelines in Ignite 3. I would identify two
> > > major goals here:
> > >
> > >1. Simplification. Having too many rules and restrictions tend to
> > >complicate development rather than providing any value. We should come
> > > up
> > >with a minimum set of rules and then make amends one by one if needed.
> > >2. The ability for automation. I hold a strong belief that code style
> > >checking has to become a part of the build. Therefore, we need to make
> > > sure
> > >that any rule that ends up in the guideline can be automatically
> > > verified.
> > >
> > > I propose the following process to define the new guideline:
> > >
> > >1. Use Google code style as the starting point:
> > >https://google.github.io/styleguide/javaguide.html
> > >2. Replace the 100 column limit with 140. The latter is the value we
> > >already use in Ignite 2, and it seems to be more reasonable, in my
> > > opinion.
> > >3. Use 4 spaces block indentation and 8 spaces for continuation (as
> > >opposed to 2 and 4). Nothing wrong with 2 spaces, in my view, but 4
> > > spaces
> > >should provide a smoother transition, as we're really used to this
> > > style.
> > >4. For any other changes, initiate separate discussions going forward.
> > >
> > > Several reasons why I specifically propose Google style:
> > >
> > >1. This is essentially the standard for many projects. I don't think
> > >there is a need for us to reinvent the wheel.
> > >2. It's minimalistic and developer-friendly. No overcomplication.
> > >3. (probably the most important) It comes with all the required tools
> > >and configurations for automation (e.g., here is the config for IDEA:
> > >
> > https://github.com/google/error-prone/blob/master/.idea/GoogleStyle.xml
> > > )
> > >
> > > Please let me know what you think. If there are no objections, I will
> > start
> > > the process.
> > >
> > > -Val
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

Igniters,

What I actually didn't understand from this thread: Is Guava allowed
in production code of Ignite 3 modules (not dependencies like
Calcite)?

While we agree with using shading I don't see any arguments about
using Guava library in our code base except for the fact that we have
some copy-paste of Guava code in the project.

Guava is rich enough library and it has both advantages and
disadvantages. Ignite code base always strived to be not overloaded
and minimalistic. For example we usually use immutability very rare,
we try to avoid complex and overloaded APIs on hot path, we try reduce
GC pressure, etc. In turn, Guava is some sense forces using of
immutable collections, has a lot of utility methods which produces
temp object (yes, I don't trust to escap analysys), etc. Rich set of
collections and utility functions will also lead to different
programming styles during Ignite development. I can't predict it, but
it is usual enough when some developers force immutablity while others
remove static type declaration and replace it by var :)

This is a common pratice to reduce language possibilies subset and
limiting using of some libraries in order to provide more or less
unified approach to coding. So I propose to disallow using Guava in
our code base.

WDYT?


On Fri, Aug 20, 2021 at 7:37 PM Alexander Polovtcev
 wrote:
>
> Guys,
>
> Thanks again for your responses. I've created a ticket about using the
> Shade plugin in order to understand if it is possible to shade and minimize
> Guava, I will start working on it shortly:
> https://issues.apache.org/jira/browse/IGNITE-15354
>
> On Tue, Aug 10, 2021 at 1:51 PM Courtney Robinson 
> wrote:
>
> > I think since Calcite brings it in already then your arguments make sense.
> > Would it be pinned to the same version as Calcite? Risk NoSuchMethodError
> > at runtime if not.
> > +1
> >
> > On Mon, Aug 9, 2021 at 9:56 AM Alexander Polovtcev <
> > alexpolovt...@gmail.com>
> > wrote:
> >
> > > Zhenya, Courtney, Andrey,
> > >
> > > What do you think about my arguments, was I able to convince you? I would
> > > like to reach some consensus here. At the moment, my original points
> > still
> > > stand, I'm also ok with shading Guava if needed, though I think it is not
> > > necessary at this point.
> > >
> > > On Fri, Aug 6, 2021 at 12:45 PM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Zhenya,
> > > >
> > > > > But there is no restrictions from running ignite server nodes from
> > some
> > > > other code with it`s own guava version seems we obtain fast path to jar
> > > > hell here?
> > > >
> > > > I'm not sure if I fully understand your question, but it looks like we
> > > are
> > > > in this situation already, because we have some dependencies that use
> > > > Guava. That's why I propose to add Guava explicitly to at least have a
> > > > deterministic runtime configuration (see this link
> > > > <
> > >
> > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management
> > > >
> > > > for an explanation).
> > > >
> > > > On Fri, Aug 6, 2021 at 12:25 PM Zhenya Stanilovsky
> > > >  wrote:
> > > >
> > > >>
> > > >> Alexander, first of all looks like Ivan Daschinsky approach about thin
> > > >> client use only and shadow plugin are cover all Andrey Mashenkov
> > listing
> > > >> problems.
> > > >> But there is no restrictions from running ignite server nodes from
> > some
> > > >> other code with it`s own guava version seems we obtain fast path to
> > jar
> > > >> hell here?
> > > >>
> > > >>
> > > >> >Zhenya,
> > > >> >
> > > >> >My intentions are the following:
> > > >> >
> > > >> >1. Remove some copy-pasted code (like the "bytecode" module or some
> > > >> utility
> > > >> >methods). Please see my original message for the links to the code.
> > > >> >2. Explicitly pin the Guava version to avoid conflicts in the
> > runtime.
> > > >> >
> > > >> >About allowing to use Guava in the codebase, my thoughts are the
> > > >> following:
> > > >> >
> > > >> >1. We *already* use some code from Guava either directly (like in the
> > > >> >"calcite" module) or by copy-pasting it into a utility class.
> > > >> >2. I understand that some Guava methods are obsolete as of Java 11,
> > but
> > > >> >some of them still don't have any standard library counterparts, in
> > > which
> > > >> >case I think using Guava is justified (which is supported by point
> > 1).
> > > >> >
> > > >> >Can you please explain why you would disapprove of my proposal?
> > > >> >
> > > >> >On Thu, Aug 5, 2021 at 7:56 PM Zhenya Stanilovsky
> > > >> >< arzamas...@mail.ru.invalid > wrote:
> > > >> >
> > > >> >>
> > > >> >> alexpolovtcev please clarify what do you mean under : «possibility
> > of
> > > >> >> using Guava in Ignite 3», using how necessary dependency of calcite
> > > or
> > > >> >> using like «using in our code» ? If using in code, i -1 here.
> > > >> >> thanks.
> > > >> >>
> > > >> >>
> > > >> >> >Hello, dear Ignit

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

Follow up

On Mon, Aug 23, 2021 at 1:22 PM Andrey Gura  wrote:
>
> Igniters,
>
> What I actually didn't understand from this thread: Is Guava allowed
> in production code of Ignite 3 modules (not dependencies like
> Calcite)?
>
> While we agree with using shading I don't see any arguments about
> using Guava library in our code base except for the fact that we have
> some copy-paste of Guava code in the project.
>
> Guava is rich enough library and it has both advantages and
> disadvantages. Ignite code base always strived to be not overloaded
> and minimalistic. For example we usually use immutability very rare,
> we try to avoid complex and overloaded APIs on hot path, we try reduce
> GC pressure, etc. In turn, Guava is some sense forces using of
> immutable collections, has a lot of utility methods which produces
> temp object (yes, I don't trust to escap analysys), etc. Rich set of
> collections and utility functions will also lead to different
> programming styles during Ignite development. I can't predict it, but
> it is usual enough when some developers force immutablity while others
> remove static type declaration and replace it by var :)
>
> This is a common pratice to reduce language possibilies subset and
> limiting using of some libraries in order to provide more or less
> unified approach to coding. So I propose to disallow using Guava in
> our code base.
>
> WDYT?
>
>
> On Fri, Aug 20, 2021 at 7:37 PM Alexander Polovtcev
>  wrote:
> >
> > Guys,
> >
> > Thanks again for your responses. I've created a ticket about using the
> > Shade plugin in order to understand if it is possible to shade and minimize
> > Guava, I will start working on it shortly:
> > https://issues.apache.org/jira/browse/IGNITE-15354
> >
> > On Tue, Aug 10, 2021 at 1:51 PM Courtney Robinson 
> > 
> > wrote:
> >
> > > I think since Calcite brings it in already then your arguments make sense.
> > > Would it be pinned to the same version as Calcite? Risk NoSuchMethodError
> > > at runtime if not.
> > > +1
> > >
> > > On Mon, Aug 9, 2021 at 9:56 AM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Zhenya, Courtney, Andrey,
> > > >
> > > > What do you think about my arguments, was I able to convince you? I 
> > > > would
> > > > like to reach some consensus here. At the moment, my original points
> > > still
> > > > stand, I'm also ok with shading Guava if needed, though I think it is 
> > > > not
> > > > necessary at this point.
> > > >
> > > > On Fri, Aug 6, 2021 at 12:45 PM Alexander Polovtcev <
> > > > alexpolovt...@gmail.com>
> > > > wrote:
> > > >
> > > > > Zhenya,
> > > > >
> > > > > > But there is no restrictions from running ignite server nodes from
> > > some
> > > > > other code with it`s own guava version seems we obtain fast path to 
> > > > > jar
> > > > > hell here?
> > > > >
> > > > > I'm not sure if I fully understand your question, but it looks like we
> > > > are
> > > > > in this situation already, because we have some dependencies that use
> > > > > Guava. That's why I propose to add Guava explicitly to at least have a
> > > > > deterministic runtime configuration (see this link
> > > > > <
> > > >
> > > https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management
> > > > >
> > > > > for an explanation).
> > > > >
> > > > > On Fri, Aug 6, 2021 at 12:25 PM Zhenya Stanilovsky
> > > > >  wrote:
> > > > >
> > > > >>
> > > > >> Alexander, first of all looks like Ivan Daschinsky approach about 
> > > > >> thin
> > > > >> client use only and shadow plugin are cover all Andrey Mashenkov
> > > listing
> > > > >> problems.
> > > > >> But there is no restrictions from running ignite server nodes from
> > > some
> > > > >> other code with it`s own guava version seems we obtain fast path to
> > > jar
> > > > >> hell here?
> > > > >>
> > > > >>
> > > > >> >Zhenya,
> > > > >> >
> > > > >> >My intent

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

> So I would propose to allow at least *some* methods that we consider useful, 
> while disallowing everything else.

It just doesn't work. If there is precedent of usage of Guava methods
then somebody will use other methods.

> I think that the benefits of using Guava methods instead of copy-pasting them 
> are quite obvious: you don't have to copy-paste code and support it

I don't understand why somebody did such a copy-paste, but I believe
that nobody supports this code. So, from my point of view, it isn't a
problem.

On Thu, Sep 2, 2021 at 2:52 PM Alexander Polovtcev
 wrote:
>
> Andrey,
> I think that the benefits of using Guava methods instead of copy-pasting them 
> are quite obvious: you don't have to copy-paste code and support it. I also 
> find this situation quite strange: we have a dependency and copy-paste code 
> from it instead of using it directly.
>
>
> On Thu, Sep 2, 2021 at 1:58 PM Andrey Mashenkov  
> wrote:
>>
>> Alex,
>>
>> As I understand we agree to shade Guava transitive dependency and
>> you said a 'maven-shade-plugin' can drop unused Guava methods to reduce the
>> footprint of Ignite jar.
>>
>> At this point, there is no difference between copy-pasting Guava method to
>> Ignite and use Guava one.
>> The resulted jar will have one copy of such method, but in the second case,
>> we have to care about compatibility when the transitive Guava dependency
>> will be updated to a new version.
>>
>> So, I see no benefits from using Guava directly.
>>
>>
>> On Thu, Sep 2, 2021 at 11:56 AM Alexander Polovtcev 
>> wrote:
>>
>> > Hi, Andrey!
>> > I mostly agree with your proposal, but, since we already have some
>> > copy-paste in our code, can we at least use Guava to remove it? So I would
>> > propose to allow at least *some* methods that we consider useful, while
>> > disallowing everything else. I understand that it may be difficult to
>> > formalize, so maybe we can create some kind of a whitelist of Guava
>> > methods? What do you think?
>> >
>> > On Mon, Aug 30, 2021 at 2:54 PM Andrey Gura  wrote:
>> >
>> > > Follow up
>> > >
>> > > On Mon, Aug 23, 2021 at 1:22 PM Andrey Gura  wrote:
>> > > >
>> > > > Igniters,
>> > > >
>> > > > What I actually didn't understand from this thread: Is Guava allowed
>> > > > in production code of Ignite 3 modules (not dependencies like
>> > > > Calcite)?
>> > > >
>> > > > While we agree with using shading I don't see any arguments about
>> > > > using Guava library in our code base except for the fact that we have
>> > > > some copy-paste of Guava code in the project.
>> > > >
>> > > > Guava is rich enough library and it has both advantages and
>> > > > disadvantages. Ignite code base always strived to be not overloaded
>> > > > and minimalistic. For example we usually use immutability very rare,
>> > > > we try to avoid complex and overloaded APIs on hot path, we try reduce
>> > > > GC pressure, etc. In turn, Guava is some sense forces using of
>> > > > immutable collections, has a lot of utility methods which produces
>> > > > temp object (yes, I don't trust to escap analysys), etc. Rich set of
>> > > > collections and utility functions will also lead to different
>> > > > programming styles during Ignite development. I can't predict it, but
>> > > > it is usual enough when some developers force immutablity while others
>> > > > remove static type declaration and replace it by var :)
>> > > >
>> > > > This is a common pratice to reduce language possibilies subset and
>> > > > limiting using of some libraries in order to provide more or less
>> > > > unified approach to coding. So I propose to disallow using Guava in
>> > > > our code base.
>> > > >
>> > > > WDYT?
>> > > >
>> > > >
>> > > > On Fri, Aug 20, 2021 at 7:37 PM Alexander Polovtcev
>> > > >  wrote:
>> > > > >
>> > > > > Guys,
>> > > > >
>> > > > > Thanks again for your responses. I've created a ticket about using
>> > the
>> > > > > Shade plugin in order to understand if it is possible to shade and
>> > > minimize
>> > > > > Guava, I wi

Re: Re[2]: Google Guava in Ignite 3

[Andrey Gura]

Alexander,

Feel free to start voting. Please, refer to this discussion in order
to avoid a new flaming thread in the voting topic.

On Thu, Sep 2, 2021 at 6:04 PM Alexander Polovtcev
 wrote:
>
> Andrey,
>
> > It just doesn't work. If there is precedent of usage of Guava methods
> then somebody will use other methods.
>
> Maybe you are right. I would suggest a vote on whether we should allow Guava 
> methods in the codebase or not. Let's do that in a separate thread? In the 
> meantime we will prohibit using Guava until the voting is complete.
>
> On Thu, Sep 2, 2021 at 3:58 PM Andrey Gura  wrote:
>>
>> > So I would propose to allow at least *some* methods that we consider 
>> > useful, while disallowing everything else.
>>
>> It just doesn't work. If there is precedent of usage of Guava methods
>> then somebody will use other methods.
>>
>> > I think that the benefits of using Guava methods instead of copy-pasting 
>> > them are quite obvious: you don't have to copy-paste code and support it
>>
>> I don't understand why somebody did such a copy-paste, but I believe
>> that nobody supports this code. So, from my point of view, it isn't a
>> problem.
>>
>> On Thu, Sep 2, 2021 at 2:52 PM Alexander Polovtcev
>>  wrote:
>> >
>> > Andrey,
>> > I think that the benefits of using Guava methods instead of copy-pasting 
>> > them are quite obvious: you don't have to copy-paste code and support it. 
>> > I also find this situation quite strange: we have a dependency and 
>> > copy-paste code from it instead of using it directly.
>> >
>> >
>> > On Thu, Sep 2, 2021 at 1:58 PM Andrey Mashenkov 
>> >  wrote:
>> >>
>> >> Alex,
>> >>
>> >> As I understand we agree to shade Guava transitive dependency and
>> >> you said a 'maven-shade-plugin' can drop unused Guava methods to reduce 
>> >> the
>> >> footprint of Ignite jar.
>> >>
>> >> At this point, there is no difference between copy-pasting Guava method to
>> >> Ignite and use Guava one.
>> >> The resulted jar will have one copy of such method, but in the second 
>> >> case,
>> >> we have to care about compatibility when the transitive Guava dependency
>> >> will be updated to a new version.
>> >>
>> >> So, I see no benefits from using Guava directly.
>> >>
>> >>
>> >> On Thu, Sep 2, 2021 at 11:56 AM Alexander Polovtcev 
>> >> 
>> >> wrote:
>> >>
>> >> > Hi, Andrey!
>> >> > I mostly agree with your proposal, but, since we already have some
>> >> > copy-paste in our code, can we at least use Guava to remove it? So I 
>> >> > would
>> >> > propose to allow at least *some* methods that we consider useful, while
>> >> > disallowing everything else. I understand that it may be difficult to
>> >> > formalize, so maybe we can create some kind of a whitelist of Guava
>> >> > methods? What do you think?
>> >> >
>> >> > On Mon, Aug 30, 2021 at 2:54 PM Andrey Gura  wrote:
>> >> >
>> >> > > Follow up
>> >> > >
>> >> > > On Mon, Aug 23, 2021 at 1:22 PM Andrey Gura  wrote:
>> >> > > >
>> >> > > > Igniters,
>> >> > > >
>> >> > > > What I actually didn't understand from this thread: Is Guava allowed
>> >> > > > in production code of Ignite 3 modules (not dependencies like
>> >> > > > Calcite)?
>> >> > > >
>> >> > > > While we agree with using shading I don't see any arguments about
>> >> > > > using Guava library in our code base except for the fact that we 
>> >> > > > have
>> >> > > > some copy-paste of Guava code in the project.
>> >> > > >
>> >> > > > Guava is rich enough library and it has both advantages and
>> >> > > > disadvantages. Ignite code base always strived to be not overloaded
>> >> > > > and minimalistic. For example we usually use immutability very rare,
>> >> > > > we try to avoid complex and overloaded APIs on hot path, we try 
>> >> > > > reduce
>> >> > > > GC pressure, etc. In turn, Guava is some sense forces using of
>> >>

Re: [VOTE] Allow or prohibit usages of the Guava library methods

[Andrey Gura]

-1
Potential compatibility issues which are related to user specific code
(services, jobs).
Potential vulnerabilities which are a problem because we can't manage
the Guava release cycle.
Arguable approach with unnecessary immutability and unmanageable GC pressure.
It's too hard to manage the *reasonable* usage of this library.

On Tue, Sep 7, 2021 at 6:31 PM Alexander Polovtcev
 wrote:
>
> +1
> We already have some utility methods that are copy-pasted from Guava, which
> can be removed, and I believe that *reasonable* usage of this library can
> benefit the code and the product quality.
>
> On Tue, Sep 7, 2021 at 6:27 PM Alexander Polovtcev 
> wrote:
>
> > Dear Igniters,
> >
> > In this thread
> > 
> > we've been discussing the problems and opportunities of using Guava
> >  in Ignite 3. We have agreed that it
> > should be added as a shaded dependency, but we haven't decided whether to
> > allow using Guava methods in the Ignite codebase or not. Therefore I would
> > like to propose a vote:
> >
> > [+1 Allow]: allow using Guava methods, if justified.
> > [-1 Prohibit]: prohibit using all Guava methods.
> >
> > The voting will commence on Monday, September 13th at 9:00 UTC. Also feel
> > free to express your opinion in the original discussion thread.
> >
> > --
> > With regards,
> > Aleksandr Polovtcev
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: Ban Java Streams usage in Ignite 3 code

[Andrey Gura]

Agree that any additional object creation on a hot path could be a
problem. So hot path should not contain stream API and any other
potentially problem code (e.g. iterator instead of for by index).

On Wed, Sep 8, 2021 at 1:45 PM Pavel Tupitsyn  wrote:
>
> Ok, maybe a total ban is overkill, but right now streams are used even on
> some hot paths like getAllAsync [1].
>
> Another issue is that Collectors.toList() and other variants don't accept
> capacity, and we end up with unnecessary reallocations of underlying arrays.
>
> [1]
> https://github.com/apache/ignite-3/blob/1d7d703ff2b18234b15a9a7b03104fbb73388edf/modules/table/src/main/java/org/apache/ignite/internal/table/KVBinaryViewImpl.java#L83
>
> On Wed, Sep 8, 2021 at 1:06 PM Konstantin Orlov  wrote:
>
> > Hi!
> >
> > Agree with Ivan that it’s an overkill. Code readability and
> > maintainability should have
> > the same priority as the performance (with some exceptions of course).
> >
> > BTW the result of your benchmark looks quite strange. The performance
> > penalty on
> > my laptop (Core i7 9750H, 6 cores up to 4.50 GHz) is 25%, not 8 times:
> >
> > Benchmark Mode  Cnt  Score Error   Units
> > JmhIncrementBenchmark.loopSumthrpt   10  32347.819 ± 676.548  ops/ms
> > JmhIncrementBenchmark.streamSum  thrpt   10  24459.196 ± 610.152  ops/ms
> >
> >
> > --
> > Regards,
> > Konstantin Orlov
> >
> >
> > > On 8 Sep 2021, at 12:23, Ivan Bessonov  wrote:
> > >
> > > Hello Igniters,
> > >
> > > I object, banning streams is an overkill. I would argue that most of the
> > > code
> > > is not on hot paths and that allocations in TLAB don't create much
> > pressure
> > > on GC.
> > >
> > > Streams must be used cautiously, developers should know whether they
> > > write hot methods or not. And if methods are not hot, code simplicity
> > must
> > > be
> > > the first priority. I don't want Ignite 3 code to look like Ignite 2
> > code,
> > > where
> > > people would iterate over Lists using explicit access by indexes,
> > because it
> > > saves them a single Iterator allocation. That's absurd.
> > >
> > > ср, 8 сент. 2021 г. в 11:43, Pavel Tupitsyn :
> > >
> > >> Igniters,
> > >>
> > >> Java streams are known to be slower and cause more GC pressure than an
> > >> equivalent loop.
> > >> Below is a simple filter/map/reduce scenario (code [1]):
> > >>
> > >> * Benchmark Mode
> > Cnt
> > >>Score Error   Units
> > >>
> > >> * StreamVsLoopBenchmark.loopSum thrpt
> >   3
> > >> 7987.016 ± 293.013  ops/ms
> > >> * StreamVsLoopBenchmark.loopSum:·gc.alloc.rate  thrpt
> >   3
> > >>   ≈ 10⁻⁴MB/sec
> > >> * StreamVsLoopBenchmark.loopSum:·gc.count   thrpt
> >   3
> > >>  ≈ 0counts
> > >>
> > >> * StreamVsLoopBenchmark.streamSum   thrpt
> >   3
> > >> 1060.244 ±  36.485  ops/ms
> > >> * StreamVsLoopBenchmark.streamSum:·gc.alloc.ratethrpt
> >   3
> > >>  315.819 ±  10.844  MB/sec
> > >> * StreamVsLoopBenchmark.streamSum:·gc.count thrpt
> >   3
> > >>   55.000counts
> > >>
> > >> Loop is several times faster and does not allocate at all.
> > >>
> > >> 1. Performance is one of the most important features of our product.
> > >> 2. Most of our APIs will be on the hot path.
> > >>
> > >> One can argue about performance differences in real-world scenarios,
> > >> but increasing GC pressure just to make the code a little bit nicer is
> > >> unacceptable.
> > >>
> > >> I propose to ban streams usage in the codebase (except for the tests).
> > >>
> > >> Thoughts, objections?
> > >>
> > >> [1] https://gist.github.com/ptupitsyn/5934bbbf8f92ac4937e534af9386da97
> > >>
> > >
> > >
> > > --
> > > Sincerely yours,
> > > Ivan Bessonov
> >
> >

Re: [VOTE] Create separate Jira project and Confluence space for Ignite 3

[Andrey Gura]

+1

On Wed, Oct 6, 2021 at 8:16 PM Pavel Tupitsyn  wrote:
>
> -1
>
> We have already started using the existing JIRA and Confluence.
> There are lots of tickets, IEPs, and commits interlinked with each other.
> Introducing a new JIRA and Confluence will cause a huge mess.
>
> On Wed, Oct 6, 2021 at 7:57 PM Denis Magda  wrote:
>
> > Val,
> >
> > As we discussed earlier, I don't fully support this idea but if what you
> > propose is the only reasonable and efficient way to carry on with Ignite 3
> > and 2 development efforts, then I trust your word. But even if this vote
> > passes I still would encourage you to experiment with the alternative - a
> > mandatory "Architecture" field in JIRA that requires to pick between
> > "Ignite 2" and "Ignite 3". Talk to the ASF INFRA team, there still might be
> > the way to keep a single JIRA project and simplify the development efforts.
> >
> > -
> > Denis
> >
> >
> > On Wed, Oct 6, 2021 at 3:03 AM Dmitriy Pavlov  wrote:
> >
> > > -0 from my side.
> > >
> > > That's fine if we have a separate project.
> > > But since the mission and the statement of community is the same, I tend
> > to
> > > see 2&3 as the same project. The only difference is that 3 was started
> > > from scratch (from the code point of view).
> > >
> > > ср, 6 окт. 2021 г. в 00:23, Maxim Muzafarov :
> > >
> > > > - 0 from me.
> > > >
> > > > This is OK if the projects are different, but not OK if they are
> > > > sharing the same version history.
> > > >
> > > > On Tue, 5 Oct 2021 at 13:39, Andrey Mashenkov
> > > >  wrote:
> > > > >
> > > > > +1
> > > > >
> > > > > вт, 5 окт. 2021 г., 13:33 Юрий :
> > > > >
> > > > > > +1
> > > > > >
> > > > > > вт, 5 окт. 2021 г. в 02:52, Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com>:
> > > > > >
> > > > > > > Hello Community,
> > > > > > >
> > > > > > > As discussed in [1], I would like to propose the creation of a
> > > > separate
> > > > > > > Jira project and Confluence space for Ignite 3.
> > > > > > >
> > > > > > > Ignite 2 and Ignite 3 are developed in parallel in separate
> > repos,
> > > > so we
> > > > > > > need a clear separation in other tools as well - this will help
> > to
> > > > > > > streamline the development process. Please refer to the
> > discussion
> > > > for
> > > > > > more
> > > > > > > details.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> > https://lists.apache.org/thread.html/rdcad3fc64b9f3a848c93089baae2bee1124a97869a94f4a04dd80fdf%40%3Cdev.ignite.apache.org%3E
> > > > > > >
> > > > > > > Voting options:
> > > > > > >
> > > > > > >- +1 - Agree with the suggestion
> > > > > > >- 0 - Don't care much about the suggestion
> > > > > > >- -1 - Disagree with the suggestion
> > > > > > >
> > > > > > > This is a majority vote.
> > > > > > >
> > > > > > > Voting ends in 72 hours, at 5pm PDT on October 7:
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> > https://www.timeanddate.com/counters/fullscreen.html?mode=a&iso=20211007T17&year=2021&month=10&day=7&hour=17&min=0&sec=0&p0=224
> > > > > > >
> > > > > > > -Val
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Живи с улыбкой! :D
> > > > > >
> > > >
> > >
> >

Re: [DISCUSSION] Add SystemRegionMetricsEnabled property on DataStorageConfiguration.

[Andrey Gura]

Hi,

Your proposal is consistent with the current interface of
DataStorageConfiguration class, so I', okay with the proposed change.

Alternatively, DataStorageConfiguration#systemDataRegionConfiguration
field could be introduced which could be DataRegionConfiguration type
or special SystemDataRegionConfiguration type in case if we want to
limit access to the system data region parameters.

On Tue, Nov 2, 2021 at 2:42 PM Eduard Rakhmankulov  wrote:
>
> Hello, Igniters!
>
> At the moment metrics on system region are disabled on node start.
> User-defined regions can be tuned to have enabled metrics via region
> configuration
> (org.apache.ignite.configuration.DataRegionConfiguration#setMetricsEnabled)
> To enable metrics on system region users should use non-public API or use
> deprecated JMX bean
> (org.apache.ignite.mxbean.DataRegionMetricsMXBean#enableMetrics) for it,
> which doesn't solve problem because it needs human attention when node
> restarts.
>
> I want to add a new property to
> *org.apache.ignite.configuration.DataStorageConfiguration* which enables or
> disables collection of metrics on system region (aka sysMemPlc) on node
> start.
>
> API change
> add new public methods:
>
>1.
>
> *org.apache.ignite.configuration.DataStorageConfiguration#isSystemRegionMetricsEnabled()*
>2.
>
> *org.apache.ignite.configuration.DataStorageConfiguration#setSystemRegionMetricsEnabled(boolean)*
>
>
> WDYT?
> --
> Best regards, Eduard.

[ANNOUNCE] Welcome Semyon Danilov as a new committer

[Andrey Gura]

Igniters,

The Apache Ignite Project Management Committee (PMC) has invited
Semyon Danilov  to become a new committer and are happy to announce
that he
has accepted.

Semyon contributes actively to the project's code base (AI  2 & 3) and
helps a lot with the development environment set up (you know, all
this Maven and plugins related issues).

Being a committer enables easier contribution to the project since there is
no need to go via the patch submission process. This should enable better
productivity.

Please join me in welcoming Ivan, and congratulating him on the new role in
the Apache Ignite Community.

Best Regards,
Andrey Gura

Re: [ANNOUNCE] Welcome Semyon Danilov as a new committer

[Andrey Gura]

I'm sorry for the mistake. Of course please join me in welcoming
**Semyon**, and congratulating him on the new role in
the Apache Ignite Community.

On Wed, Dec 1, 2021 at 8:51 AM Zhenya Stanilovsky
 wrote:
>
>
> Wellcome Semyon !
> Andrey what`s Ivan are you talking at the end of the message or this is some 
> kind of phraseologism that all russians are ivans ?:)
>
> >Igniters,
> >
> >The Apache Ignite Project Management Committee (PMC) has invited
> >Semyon Danilov to become a new committer and are happy to announce
> >that he
> >has accepted.
> >
> >Semyon contributes actively to the project's code base (AI 2 & 3) and
> >helps a lot with the development environment set up (you know, all
> >this Maven and plugins related issues).
> >
> >Being a committer enables easier contribution to the project since there is
> >no need to go via the patch submission process. This should enable better
> >productivity.
> >
> >Please join me in welcoming Ivan, and congratulating him on the new role in
> >the Apache Ignite Community.
> >
> >Best Regards,
> >Andrey Gura
>
>
>
>

Re: [VOTE] Release Apache Ignite 3.0.0-alpha4 RC1

[Andrey Gura]

+1 (binding)

On Wed, Jan 26, 2022 at 4:46 PM Denis Magda  wrote:
>
> +1 (binding)
>
> —
> Denis
>
> > On Jan 25, 2022, at 2:44 PM, Valentin Kulichenko 
> >  wrote:
> >
> > Dear Community,
> >
> > Ignite 3 is moving forward and I think we're in a good spot to release
> > another alpha version. In the last few month the following major features
> > have been added:
> >
> >   - Transactional API
> >   - Record and KeyValue views with POJO mapping support
> >   - DDL
> >
> > This is a significant milestone, so I think it will be valuable to share
> > the current state with the broader community. I propose to release 3.0.0
> > Alpha4 with the features listed above.
> >
> > Please vote.
> >
> > ---
> >
> > Release candidate:
> > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha4-rc1/
> > Maven staging:
> > https://repository.apache.org/content/repositories/orgapacheignite-1540/
> > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha4-rc1
> >
> > +1 - accept Apache Ignite 3.0.0-alpha4 RC1
> > 0 - don't care either way
> > -1 - DO NOT accept Apache Ignite 3.0.0-alpha4 RC1 (explain why)
> >
> > Voting guidelines: https://www.apache.org/foundation/voting.html
> > How to verify the release: https://www.apache.org/info/verification.html
> >
> > The vote will be open for 72 hours and will close on January 28th at 8pm
> > UTC: https://www.timeanddate.com/countdown/to?iso=20220128T12&p0=224
> >
> > -Val

[ANNOUNCE] New wiki section for Apache Ignite 3 contributors

[Andrey Gura]

Igniters,

The new wiki section for Apache Ignite 3 contributors is created. This
page [1] is the main entry point for new and existing contributors and
the page provides information about differences in the development
process (e.g. code style, practices, etc), the product architecture,
technical details, etc.

At the moment the Java Code Style Guide page [2] contains information
about Apache Ignite 3 code style and some additional rules discussed
on dev-list.

Please, feel free to contribute appropriate content to this wiki
section. It would be great to have pages about project setup, best
coding practices for developers, etc.

1. https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+3
2. https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide

[DISCUSSION] IEP-86: Colocation Key (Apache Ignite 3.0)

[Andrey Gura]

Igniters,

I'd like to start a discussion about API changes related to the data
colocation in Apache Ignite 3.0. [1]

The main purpose of the proposal is a definition of tables' colocation
from a user perspective including DDL syntax and changes related to
used terminology.

While proposed changes are already implemented partially we still can
discuss these changes and revisit some aspects.


[1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-86:+Colocation+Key

Re: [VOTE] Release Extensions Ignite Spring Data 2.2.0 RC1, Ignite Spring Session 1.0.0 RC1, Ignite Zookeeper IP Finder 1.0.0 RC1

[Andrey Gura]

+1

On Fri, Apr 29, 2022 at 7:22 PM Nikita Amelchev  wrote:
>
> +1
>
> пт, 29 апр. 2022 г. в 17:54, Anton Vinogradov :
> >
> > +1
> >
> > On Fri, Apr 29, 2022 at 5:28 PM Maxim Muzafarov  wrote:
> >
> > > Dear Community,
> > >
> > >
> > > The release candidates are ready. See the links below.
> > >
> > >
> > > == Ignite Spring Data Extension Module 2.2.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-data-ext-2.2.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1547/org/apache/ignite/ignite-spring-data-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-spring-data-ext-2.2.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-spring-data-ext-2.2.0/modules/spring-data-ext/spring-data/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6229346844?check_suite_focus=true
> > >
> > >
> > >
> > > == Ignite Spring Session Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spring-session-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1548/org/apache/ignite/ignite-spring-session-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-spring-session-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-spring-session-ext-1.0.0/modules/spring-session-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6229613223?check_suite_focus=true
> > >
> > >
> > > == Ignite Zookeeper Ip Finder Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-zookeeper-ip-finder-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1549/org/apache/ignite/ignite-zookeeper-ip-finder-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-zookeeper-ip-finder-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-zookeeper-ip-finder-ext-1.0.0/modules/zookeeper-ip-finder-ext/zookeeper-ip-finder/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6210193576?check_suite_focus=true
> > >
> > >
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept (Ignite Spring Data 2.2.0 RC1, Ignite Spring Session
> > > 1.0.0 RC1, Ignite Zookeeper IP Finder 1.0.0 RC1)
> > > 0 - don't care either way
> > > -1 - DO NOT accept (explain why)
> > >
> > > See notes on how to verify release here
> > > https://www.apache.org/info/verification.html
> > > and
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> > >
> > >
> > > This vote will be open for at least 7 days till Sat May 7, 2022,
> > > 14:00 Moscow TZ. Please, write me down the thread if you need
> > > additional time to check the release.
> > >
> > >
> > > https://www.timeanddate.com/countdown/vote?iso=20220507T14&p0=166&msg=%5BVOTE%5D+Release+Extensions+Ignite+Spring+Data+2.2.0+RC1%2C+Ignite+Spring+Session+1.0.0+RC1%2C+Ignite+Zookeeper+IP+Finder+1.0.0+RC1&font=sanserif
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita

Re: [VOTE] Release Ignite AWS 1.0.0 RC1, Ignite Azure 1.0.0 RC1, Ignite GCE 1.0.0 RC1

[Andrey Gura]

+1

On Fri, Apr 29, 2022 at 7:22 PM Nikita Amelchev  wrote:
>
> +1
>
> пт, 29 апр. 2022 г. в 15:51, Anton Vinogradov :
> >
> > +1
> >
> > On Fri, Apr 29, 2022 at 3:48 PM Maxim Muzafarov  wrote:
> >
> > > Dear Community,
> > >
> > >
> > > The release candidates are ready. See the links below.
> > >
> > >
> > > == Ignite AWS Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-aws-ext-1.0.0-rc1
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1543/org/apache/ignite/ignite-aws-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-aws-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-aws-ext-1.0.0/modules/aws-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6227581198?check_suite_focus=true
> > >
> > >
> > > == Ignite Azure Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-azure-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1545/org/apache/ignite/ignite-azure-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-azure-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-azure-ext-1.0.0/modules/azure-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6228019576?check_suite_focus=true
> > >
> > >
> > > == Ignite GCE Extension Module 1.0.0 ==
> > >
> > > The release candidate is uploaded to:
> > >
> > > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-gce-ext-1.0.0-rc1/
> > >
> > > The following Maven staging can be used:
> > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1546/org/apache/ignite/ignite-gce-ext/
> > >
> > > Tag:
> > >
> > > https://github.com/apache/ignite-extensions/releases/tag/ignite-gce-ext-1.0.0-rc1
> > >
> > > RELEASE_NOTES:
> > >
> > > https://github.com/apache/ignite-extensions/blob/release/ignite-gce-ext-1.0.0/modules/gce-ext/RELEASE_NOTES.txt
> > >
> > > DEVNOTES:
> > > https://github.com/apache/ignite-extensions/blob/master/DEVNOTES.md
> > >
> > > Release Checker Results:
> > >
> > > https://github.com/apache/ignite-extensions/runs/6228169980?check_suite_focus=true
> > >
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept (Ignite AWS 1.0.0 RC1, Ignite Azure 1.0.0 RC1, Ignite
> > > GCE 1.0.0 RC1)
> > > 0 - don't care either way
> > > -1 - DO NOT accept (explain why)
> > >
> > > See notes on how to verify release here
> > > https://www.apache.org/info/verification.html
> > > and
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> > >
> > >
> > > This vote will be open for at least 7 days till Fri May 7, 2022,
> > > 14:00 Moscow TZ. Please, write me down the thread if you need
> > > additional time to check the release.
> > >
> > >
> > > https://www.timeanddate.com/countdown/vote?iso=20220507T14&p0=166&msg=%5BVOTE%5D+Release+Ignite+AWS+1.0.0+RC1%2C+Ignite+Azure+1.0.0+RC1%2C+Ignite+GCE+1.0.0+RC1&font=sanserif
> > >
>
>
>
> --
> Best wishes,
> Amelchev Nikita

Re: [DISCUSSION] Update documentation with GitHub Actions

[Andrey Gura]

+1

Great! Thanks!

On Sun, May 1, 2022 at 8:51 AM Pavel Tupitsyn  wrote:
>
> +1, this is awesome
>
> On Fri, Apr 29, 2022 at 3:26 PM Nikita Safonov 
> wrote:
>
> > +1
> >
> > Cannot but support!
> >
> > Thanks,
> > Nikita
> >
> > сб, 23 апр. 2022 г. в 17:47, Maxim Muzafarov :
> >
> > > Hello,
> > >
> > > +1, great improvement.
> > >
> > > On Sat, 23 Apr 2022 at 17:26, Nikita Amelchev 
> > > wrote:
> > > >
> > > > Hello, Igniters.
> > > >
> > > > I propose to add a GitHub action to update documentation for released
> > > > Ignite versions by a click. [1]
> > > >
> > > > For now, this is a complex manual work that requires understanding all
> > > > the intermediate steps. [2]
> > > >
> > > > New process to update documentation:
> > > > 1. Commit documentation changes to a released branch.
> > > > 2. Run the action with a click.
> > > > 3. Check the result.
> > > >
> > > > Moreover, this will also simplify the process of publishing a new
> > > release.
> > > >
> > > > ASF's GitHub Actions Policy allows automated services to push changes
> > > > related to documentation. [3]
> > > > Write access is required to run the workflow. So, only committers can
> > > > run the action. [4]
> > > >
> > > > WDYT? Any objections?
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-16895
> > > > [2]
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=85461527#HowtoDocument-UpdatingPublishedDocs
> > > > [3] https://infra.apache.org/github-actions-policy.html
> > > > [4]
> > >
> > https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
> > > >
> > > > --
> > > > Best wishes,
> > > > Amelchev Nikita
> > >
> >

Re: Ignite 3 IEP

[Andrey Gura]

Aleksandr,

could you please create an Apache Confluence account [1] and provide
your user name?

1. https://cwiki.apache.org/confluence/signup.action

On Thu, May 5, 2022 at 7:24 PM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters. I am working on Ignite 3 REST server and a new CLI tool. I’d 
> like to develop IEPs both for REST and CLI. Could you please allow me to add 
> new pages to the Confluence page 
> https://cwiki.apache.org/confluence/display/IGNITE/Proposals+for+Ignite+3.0 ? 
>  My email is apk...@gmail.com.
>
> Best regards,
> Aleksandr Pakhomov

Re: Ignite 3 IEP

[Andrey Gura]

Permissions are granted. Please, check.

On Fri, May 6, 2022 at 1:32 PM Aleksandr Pakhomov  wrote:
>
> Sure,
>
> apkhmv
>
> > On 6 May 2022, at 13:05, Andrey Gura  wrote:
> >
> > Aleksandr,
> >
> > could you please create an Apache Confluence account [1] and provide
> > your user name?
> >
> > 1. https://cwiki.apache.org/confluence/signup.action
> >
> > On Thu, May 5, 2022 at 7:24 PM Aleksandr Pakhomov  wrote:
> >>
> >> Hello, Igniters. I am working on Ignite 3 REST server and a new CLI tool. 
> >> I’d like to develop IEPs both for REST and CLI. Could you please allow me 
> >> to add new pages to the Confluence page 
> >> https://cwiki.apache.org/confluence/display/IGNITE/Proposals+for+Ignite+3.0
> >>  ?  My email is apk...@gmail.com.
> >>
> >> Best regards,
> >> Aleksandr Pakhomov
>

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Aleksandr,

it would be better to finish the IEP draft and then start a discussion
thread on dev list. At the moment I see TBD sections and it makes
impossible to have a full view about the proposal.

Thanks!

On Fri, May 6, 2022 at 6:11 PM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters.
>
> I'd like to add a few 3rd party libraries to Ignite 3 Java Code Style Guide 
> [1]. As I mentioned in IEP-87 [2], it will be:
> Micrionaut for REST [3]
> Swagger for Open API annotations [4]
> Micronaut serde for REST serialization [5]
>
> WDYT? Any objections? Also, comments on IEP-87 are welcomed.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> [2] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> [3] https://micronaut.io/docs/
> [4] https://swagger.io
> [5] 
> https://micronaut-projects.github.io/micronaut-serialization/snapshot/guide/
>
> --
> Best regards,
> Aleksandr Pakhomov

Re: [DISCUSSION] IEP-87: Open API support for REST

[Andrey Gura]

Hi,

I took a look at the proposal and have some questions and comments.

1. It is not clear what the main value of this proposal is. The
current implementation of REST is code-first. API specification could
be written manually. It seems that the main value is the possibility
to generate an API specification from code. If so, then it would be
great to point it out strongly. Otherwise, this proposal looks like an
attempt to replace one implementation by another one (may be more
popular) but with additional 3rd party dependencies. Also, it would be
great to propose a process of artefact management (what should we do
and when) in relation to specification (Where it should be published?
Should it be placed in a source code repository or not? Should we do
some version management?)

2. The "Modular architecture support" part says: "Ignite modules can
provide endpoints that will be included into the netty server by
RestComponent **at the build time**". If I understood correctly, we
talk only about endpoints here, but registration/deregistration of
handlers for known endpoints could be done at runtime. Am I right?

3. The "API" part refers to the meta storage nodes and CMG nodes.
Could you please refer to a document which introduces these concepts?

4. Also it would be great to state that the proposed API actually is
the current API which exists in Apache Ignite.

5. The task about developer documentation should be added to the
issues list. The documentation is a readme.md file which will help an
Ignite developer to understand how to add a new endpoint, how to
generate API specification, etc.

On Wed, May 11, 2022 at 11:17 AM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters.
>
> I’d like to start a discussion about Open API support for REST [1]. The main 
> purpose of this improvement is to add the support of Open API specification 
> by generating it from the source code.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST

Re: [DISCUSSION] IEP-88: CLI Tool

[Andrey Gura]

Hi,

My two cents:

1. CLI Tool - looks like not the best name :) Shell?

2. The description says: "REPL mode is used by default and is
activated if the ignite command is executed without parameters."

I think it is a bad idea. Firstly, it is usual CLI's behaviour to
print a help for a user. An exception if we would use a special CLI
for REPL. Shell? :)
Also such approach breaks my expectations related to a scripting. In
case of an error in a script it will just hang. So, a separate CLI
utility looks better than ignite CLI.


On Thu, May 12, 2022 at 11:43 PM Aleksandr Pakhomov  wrote:
>
> Hello, Igniters.
>
> I’d like to start a discussion about Ignite 3 Command Line Tool [1]. The main 
> value is to develop a user-friendly command-line tool with advanced 
> completions and SQL REPL mode.
>
> The set of commands and parameters can be discussed. Questions and comments 
> are welcomed.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool 
> 

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

I personally don't support any additional 3rd party dependencies as
well as I don't fully understand the value of autogenerated specs for
REST endpoints. In my opinion we have another option: writing spec
manually. This option doesn't require any of proposed dependencies and
allow to avoid possible vulnerabilities. Of course at the cost of
manual actions.

I understand that my statement is arguable. So I'll just wait for
opinions of other community members.

On Mon, May 16, 2022 at 7:45 PM Aleksandr Pakhomov  wrote:
>
>
> Hello, Igniters.
>
> I would like to start a discussion about Java Code Style Guide [1] changes 
> that are going to be a part of IEP-87 [2] implementation. The set of 
> libraries and frameworks that are going to be allowed to be used in 
> production:
> - Micronaut for REST Server [3]
> - Swagger for Open API annotations [4]
> - Micronaut serde for REST serialization [5]
>
> Any objections? Also, comments on IEP-87 are welcomed.
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>  
> 
> [2] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
>  
> 
> [3] https://micronaut.io/docs /
> [4] https://swagger.io 
> [5] 
> https://micronaut-projects.github.io/micronaut-serialization/snapshot/guide 
> /

Re: IEP-90 Ignite 3 Client Lifecycle

[Andrey Gura]

Igor,

Thanks for the proposal.

I understand that such a situation is almost impossible but "if
anything can go wrong, it will". Does the protocol take into account
that different connections on different client instances theoretically
could generate an already existing connection ID?

Also, do I understand correctly that a server has enough information
about client connections so it will be possible to observe a
connections list on the server? It would be useful for cluster
monitoring purposes.

On Tue, May 17, 2022 at 3:11 PM Igor Sapego  wrote:
>
> 1. I've tried to clarify IDs part;
> 2. Maybe you are right, though in this case we'd need to use authentication
> in ConnectionRestoreReq. Which sounds reasonable to me.
>
> Best Regards,
> Igor
>
>
> On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn 
> wrote:
>
> > Igor,
> >
> > The proposal looks good to me. Very detailed!
> >
> > A couple comments:
> >
> > 1. There is a bit of a term mixup with "Connection ID", "Node ID", "Token"
> > - can you please review those?
> >
> > 2. > The Connection ID should be generated using a proper secure algorithm
> > (additional research is required here) to make sure an intruder can not
> > generate an existing Connection ID
> > Not sure about the reasoning here. I think randomUUID() should be enough:
> > - In the case of an unsecured cluster it does not matter, because anyone
> > can do anything.
> > - In the case of a secured cluster it does not matter, because
> > authentication/authorization keeps intruders out.
> >
> >
> > On Mon, May 16, 2022 at 11:07 PM Igor Sapego  wrote:
> >
> > > Hi, Igniters
> > >
> > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main idea is
> > to
> > > define client lifecycle as well as core algorithms and mechanisms used by
> > > clients. This proposal can be used as a reference for implementation of a
> > > new client for Ignite when dealing with such problems as:
> > >
> > >- Resolving of user-provided addresses;
> > >- Initial connection to a cluster;
> > >- Maintaining cluster connection;
> > >- Connection recovery;
> > >- Connection break handling.
> > >
> > > So take a look and let me know what you think guys.
> > >
> > > [1] -
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-90+Client+Lifecycle
> > >
> > > Best Regards,
> > > Igor
> > >
> >

Re: [DISCUSSION] IEP-88: CLI Tool

[Andrey Gura]

> 1. Does “Ignite shell” sounds better?

I don't know. It could be Ignite REPL, IGnite Shell or something else.
Just an idea.

> But I did not get your point about scripting. What do you mean by saying “it 
> will just hang”?  What kind of error is going to be the cause of such 
> behaviour?

I can just have a mistake in my script, e.g. running ignite command
without any parameters. What will happen in such a case from the
script perspective? I think the script will wait for returning value
while the shell will wait for a user input. Due to a server-side
nature of the script it will hang forever because there is no user on
the server side.

On Thu, May 19, 2022 at 1:02 AM Aleksandr Pakhomov  wrote:
>
> Hi, Andrey
>
> Thanks for comments.
>
> 1. Does “Ignite shell” sounds better?
>
> 2. Yes, you are right. Most tools print help if they are executed without 
>   arguments. But I did not get your point about scripting. What do you mean 
> by saying “it will just hang”?  What kind of error is going to be the cause 
> of such behaviour?
>
> > On 18 May 2022, at 21:14, Andrey Gura  wrote:
> >
> > Hi,
> >
> > My two cents:
> >
> > 1. CLI Tool - looks like not the best name :) Shell?
> >
> > 2. The description says: "REPL mode is used by default and is
> > activated if the ignite command is executed without parameters."
> >
> > I think it is a bad idea. Firstly, it is usual CLI's behaviour to
> > print a help for a user. An exception if we would use a special CLI
> > for REPL. Shell? :)
> > Also such approach breaks my expectations related to a scripting. In
> > case of an error in a script it will just hang. So, a separate CLI
> > utility looks better than ignite CLI.
> >
> >
> > On Thu, May 12, 2022 at 11:43 PM Aleksandr Pakhomov  
> > wrote:
> >>
> >> Hello, Igniters.
> >>
> >> I’d like to start a discussion about Ignite 3 Command Line Tool [1]. The 
> >> main value is to develop a user-friendly command-line tool with advanced 
> >> completions and SQL REPL mode.
> >>
> >> The set of commands and parameters can be discussed. Questions and 
> >> comments are welcomed.
> >>
> >> [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool 
> >> <https://cwiki.apache.org/confluence/display/IGNITE/IEP-88:+CLI+Tool>
>

Re: IEP-90 Ignite 3 Client Lifecycle

[Andrey Gura]

Thanks for the answers, Igor.

On Thu, May 19, 2022 at 10:55 PM Igor Sapego  wrote:
>
> Andrey,
>
> 1. If a server generates a UUID that already exists it can check and just
> re-generate it straight away
> as a check is just a simple map lookup.
>
> 2. Well, yes. This proposal does not consider monitoring, but with current
> approach it will be definitely
> easier to implement it properly.
>
> Pavel,
>
> Wow, that's really cool that you've already started working on it.
>
> Regarding your proposal I like it - it will really make a procedure easier
> and faster for a client. I'll change
> the IEP accordingly.
>
> Best Regards,
> Igor
>
>
> On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn 
> wrote:
>
> > Igor,
> >
> > I've started working on the initial sessions implementation [1],
> > and I'd like to clarify the procedure of logical connection restore.
> >
> > > client in its turn tries to establish a new *node connection*
> > > and restore *logical connection *using ConnectionRestoreReq
> >
> > This implies either an additional request, or something that replaces
> > normal handshake.
> > However, we need to handle two cases here:
> > - Logical connection is restored
> > - Logical connection is not restored (timed out, server restarted, etc), in
> > which case we still establish the connection and use it.
> >
> > To account for the second case, we should always start with a regular
> > handshake.
> > I propose to add a nullable UUID field to the handshake request to cover
> > both cases:
> > - connectionId is null or not found on server: proceed with normal
> > handshake, return newly generated connectionId.
> > - connectionId is not null and found on server: restore logical connection,
> > return the same connectionId
> >
> > Client checks if returned connectionId equals to the original and
> > understands whether logical connection was restored or not.
> >
> > Thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-16928
> >
> > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn 
> > wrote:
> >
> > > Andrey,
> > >
> > > > different connections on different client instances theoretically
> > > > could generate an already existing connection ID
> > >
> > > Do you mean an UUID collision?
> > >
> > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura  wrote:
> > >
> > >> Igor,
> > >>
> > >> Thanks for the proposal.
> > >>
> > >> I understand that such a situation is almost impossible but "if
> > >> anything can go wrong, it will". Does the protocol take into account
> > >> that different connections on different client instances theoretically
> > >> could generate an already existing connection ID?
> > >>
> > >> Also, do I understand correctly that a server has enough information
> > >> about client connections so it will be possible to observe a
> > >> connections list on the server? It would be useful for cluster
> > >> monitoring purposes.
> > >>
> > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego  wrote:
> > >> >
> > >> > 1. I've tried to clarify IDs part;
> > >> > 2. Maybe you are right, though in this case we'd need to use
> > >> authentication
> > >> > in ConnectionRestoreReq. Which sounds reasonable to me.
> > >> >
> > >> > Best Regards,
> > >> > Igor
> > >> >
> > >> >
> > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn  > >
> > >> > wrote:
> > >> >
> > >> > > Igor,
> > >> > >
> > >> > > The proposal looks good to me. Very detailed!
> > >> > >
> > >> > > A couple comments:
> > >> > >
> > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID",
> > >> "Token"
> > >> > > - can you please review those?
> > >> > >
> > >> > > 2. > The Connection ID should be generated using a proper secure
> > >> algorithm
> > >> > > (additional research is required here) to make sure an intruder can
> > >> not
> > >> > > generate an existing Connection ID
> > >> > > Not sure about the reasoning here. I think randomUUID

Apache Ignite 3.0.0 alpha 5 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Hi Igniters,

Four months have passed already since the Ignite 3 alpha 4 release. At
the moment we have a set of features that can be released in order to
give to a user an ability to try the features and share some feedback
with the community. The expected feature list consists of:

  - Pluggable storages: ability to choose a specific storage for a
table (LSM based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improves UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API (design only) for executing SQL
queries on a cluster.

I want to propose myself to be the release manager of the Ignite 3 alpha 5.

Also I propose the following milestones for the release:

Scope Freeze: June 1, 2022
Code Freeze: June 3, 2022
Voting Date: June 6, 2022
Release Date: June 10, 2012

Please, take into account that the proposed release is still alpha, so
we can afford to have such a compressed schedule.

WDYT?

Re: [DISCUSS] Query API for Ignite 3.0

[Andrey Gura]

Andrey,

there is no IEP for Java API for SQL and would be great to have it.
I'll create the IEP soon.

On Fri, May 20, 2022 at 1:56 PM Andrey Mashenkov
 wrote:
>
> JFYI. we have merged the initial version of SQL public API [1] and are
> going to implement it in epic [2] and I've created few tickets on this.
>
> Andey Gura, Taras Ledkov, Konstantine Orlov would you mind to share your
> thoughts on implementation part in some IEP?
> Is there IEP page created on the topic?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15212
> [2] https://issues.apache.org/jira/browse/IGNITE-16952
>
> On Tue, Apr 6, 2021 at 4:23 PM Andrey Mashenkov 
> wrote:
>
> > Hi Taras.
> >
> > 1. AFAIK, only Thin clients will be available in 3.0.
> > However, yes, Native JDBC API is still "must have" on servers.
> >
> > 2. We won't have other projects in dependencies if possible. Unless
> > we/they are Jigsaw compatible though.
> >
> > 2.1  I think it makes sense to be an R2DBC-compatible as it is a widely
> > used framework. E.g. Spring supports R2DBC [1].
> > Having an extension with R2DBC support will make Spring integration with
> > Ignite easy in the future.
> >
> > If it will be enough to have a dependency on just reactive-streams API [2],
> > then adding this API to dependencies looks ok to me, as there are only a
> > few interfaces and converters to/from JDK Flow API.
> > And the question is would we go with Flow API or Reactive-Streams?
> >
> > If we need some more than reactive-streams API, then I would suggest to go
> > with Flow API.
> >
> > 2.2 ADBA is compatible with R2DBC but was discontinued [3].
> >
> > 2.3 I don't think custom API is ever useful. We will need
> > converters/adapters to Flow or Reactive-streams for easy integration with
> > other APIs/frameworks.
> > Otherwise, uses will have difficulties interacting with Ignite in a
> > reactive way.
> >
> > [1]
> > https://spring.io/blog/2018/12/07/reactive-programming-and-relational-databases
> > [2] https://github.com/reactive-streams/reactive-streams-jvm
> > [3]
> > https://mail.openjdk.java.net/pipermail/jdbc-spec-discuss/2019-September/000529.html
> >
> > On Tue, Apr 6, 2021 at 3:05 PM Taras Ledkov  wrote:
> >
> >> Hi,
> >>
> >> Ignite 3.0 requires a rethinking of the query API.
> >> We have 'cache.query()' and 'SqlFieldQuery' abstractions at the Ignite
> >> 2.x native API and several JDBC implementation for clients.
> >>
> >> I propose to think about new query/SQL API for the Ignite 3.0.
> >>
> >> My vision is something like this:
> >> Ignite will support two query APIs: standard JDBC (on native) and one of
> >> reactive DB API.
> >>
> >> 1. Native JDBC API, e.g.:
> >>  Connection conn = node.sql().connection(props);
> >>
> >> JDBC is the industrial standard of the DB access and we have to support
> >> one.
> >> Also:
> >> 1.1. Thin JDBC client will be really thin: provide network communication
> >> layer and transparently map to native API.
> >> 1.2. Thick JDBC client implementation will be trivial: start client node
> >> and open JDBC connection on the started node.
> >> 1.3. JDBC provides sufficient functionality to implement ODBC (need to
> >> investigate: may be thin protocol may be extended to unify JDBC and ODBC).
> >>
> >> 2. About reactive DB API.
> >> I don't know of any industrial standard API for DB reactive access now.
> >> There are several candidates:
> >> 2.1. R2DBC look like the popular and alive. See [1];
> >> 2.2. ADBA (java.sql2 [2]) looks like not alive. Not sure;
> >> 2.3. Custom async/reactive API.
> >> e.g. oracle DB use this way [3].
> >>
> >> Igniters, WDYT?
> >>
> >> [1]. https://github.com/r2dbc/r2dbc-spi
> >> [2].
> >> http://cr.openjdk.java.net/~lancea/apidoc/java/sql2/package-summary.html
> >> [3].
> >>
> >> https://docs.oracle.com/en/database/oracle/oracle-database/21/jjdbc/jdbc-reactive-extensions.html#GUID-1C40C43B-3823-4848-8B5A-D2F97A82F79B
> >>
> >> --
> >> Taras Ledkov
> >> Mail-To: tled...@gridgain.com
> >>
> >>
> >
> > --
> > Best regards,
> > Andrey V. Mashenkov
> >
>
>
> --
> Best regards,
> Andrey V. Mashenkov

Re: [DISCUSSION] IEP-88: CLI Tool

[Andrey Gura]

Aleksandr,

Both proposed options look good to me because both cases assume that a
user must express their intent explicitly.

On Thu, May 19, 2022 at 10:53 AM Aleksandr Pakhomov  wrote:
>
> I got it. What do you think about this proposal:
>
> -  “ignite”  prints help
> -  “ignite shell” enters REPL
>
> Or
>
> -  “ignite” prints help
> -  “ignite-shell” enters REPL and it is a separate application
>
> I prefer the first varian but I would like to hear opinions of other 
> community members.
>
>
> > On 19 May 2022, at 01:16, Andrey Gura  wrote:
> >
> > I can just have a mistake in my script, e.g. running ignite command
> > without any parameters. What will happen in such a case from the
> > script perspective? I think the script will wait for returning value
> > while the shell will wait for a user input. Due to a server-side
> > nature of the script it will hang forever because there is no user on
> > the server side.
>

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

> The main
functionality of micronaut framework is REST, so this
library is scanned for vulnerabilities regularly and fixes
them asap (looking to [1] it takes about a week).
I don't  think that Ignite REST server implementation
will be scanned as regular as micronaut and issues
will be fixed as quickly as micronaut does.

The problem is not the scan frequency. The actual problem is the
release cycle: somebody should fix a vulnerability in some dependency,
then that dependency should be released and in the end Ignite should
refresh dependencies and then should be released.

In absence of dependencies we just fix the problem and build a
maintenance release.

> As for autogenerated spec, I would mention that manual
spec writing introduces the second source of truth for
the API. So, implementation declares one API, spec
declares another API and there is no prooved contract
between them.

>From my point of view this is a bogus problem. Of course the described
situation is possible and often happens. But the presence of a
generation step does not protect against problems on other levels. I
can describe the interface correctly but implementation of this
interface can be wrong. So such instruments give just an illusion of
correctness. Where is the truth? In fact, no one knows.

On Fri, May 20, 2022 at 3:39 PM Aleksandr Pakhomov  wrote:
>
> Hi Andrey,
>
> Thank you for the valuable arguments.
>
> Speaking about micronaut, it is a popular library that
> provides a lot of build-in features like error handling,
> auth, IoC, test infrastructure, and many more. The main
> functionality of micronaut framework is REST, so this
> library is scanned for vulnerabilities regularly and fixes
> them asap (looking to [1] it takes about a week).
> I don't  think that Ignite REST server implementation
> will be scanned as regular as micronaut and issues
> will be fixed as quickly as micronaut does.
>
> As for autogenerated spec, I would mention that manual
> spec writing introduces the second source of truth for
> the API. So, implementation declares one API, spec
> declares another API and there is no prooved contract
> between them. For example, a developer adds "name"
> parameter to the existing endpoint and goes to the
> spec and adds "names" parameter there (makes a mistake).
> What is the right parameter here "name" or "names"?
> Also, if there won't be a compatibility test this mistake could
> go to the production and API spec consumers could get
> a REST client that is not compatible with the server.
>
>
> > On 19 May 2022, at 00:32, Andrey Gura  wrote:
> >
> > I personally don't support any additional 3rd party dependencies as
> > well as I don't fully understand the value of autogenerated specs for
> > REST endpoints. In my opinion we have another option: writing spec
> > manually. This option doesn't require any of proposed dependencies and
> > allow to avoid possible vulnerabilities. Of course at the cost of
> > manual actions.
> >
> > I understand that my statement is arguable. So I'll just wait for
> > opinions of other community members.
>

Re: [VOTE] Add micronaut dependency to Ignite 3

[Andrey Gura]

-1 (binding) from me.

Because (if I understood correctly) the main value of the IEP-87 is
the possibility to generate API specification and Swagger annotations
is enough for this purpose I don't see reasons for these dependencies.
We already have our own controllers for REST-like API's
implementation. Why can't we just use Swagger annotations only in
addition to our rest-api module?

On Mon, May 23, 2022 at 8:08 PM Aleksandr Pakhomov  wrote:
>
> Dear community,
>
> Micronaut-based REST server implementation was a hot
> topic we discussed in the previous week. So, I've separeted
> votes about swagger and micronaut. This vote is about
> adding micronaut to the Ignite 3.
>
> The exact list of dependencies could be fined in IEP-87 [1]
> io.micronaut.serde:micronaut-serde
> io.micronaut:micronaut-context
> io.micronaut:micronaut-http
> io.micronaut:micronaut-inject
> io.micronaut:micronaut-http-server
> io.micronaut:micronaut-runtime
> io.micronaut:micronaut-core
> io.micronaut:micronaut-http-server-netty
> io.micronaut:micronaut-http-netty
> io.micronaut:micronaut-buffer-netty
> io.micronaut:micronaut-aop
> io.micronaut:micronaut-core-reactive
> Io.micronaut:micronaut-json-core
> io.micronaut:micronaut-jackson-core
>
> Swagger is out of the scope of this voting.
>
> The vote is formal, see voting guidelines [2]
>
> +1 - to accept additional dependencies to be included to Java code Guidelines 
> [3]
> 0 - don't care either way
> -1 - DO NOT accept (explain why)
>
> This vote will be open for at least 4 days till Fri May 27, 2022,
> 21:00 Moscow TZ.
>
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> [2] https://www.apache.org/foundation/voting.html 
> 
> [3] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>  
> 
> [4] 
> https://www.timeanddate.com/countdown/generic?iso=20220527T21&p0=166&msg=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3&font=cursive

Re: [VOTE] Add swagger dependency to Ignite 3

[Andrey Gura]

Ilya,

are there any alternatives to Swagger that you could recommend that
don't have the mentioned drawback?

It seems that OPen API itself doesn't define primitive and wrapped
types because such information is language/runtime/etc specific. Maybe
this problem will be addressed in the future.

On Mon, May 23, 2022 at 8:55 PM Ilya Kasnacheev
 wrote:
>
> Hello!
>
> Back when I looked at it, Swagger was very primitive, such as not
> supporting primitive types in generated models
> https://stackoverflow.com/a/45053804/36498
>
> I'm not sure it is the right tool, please clarify why it is needed.
>
> -0.5 from me (binding)
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 23 мая 2022 г. в 19:57, Aleksandr Pakhomov :
>
> > Dear community,
> >
> > Discussion about 3rd party dependencies took place
> > and I think it is time to vote if we agreed to include
> > swagger dependency to the Ignite 3 or not.
> >
> > The exact list of dependencies could be fined in IEP-87 [1]
> > (swagger-annotations, swagger-core,
> > swagger-codegen-maven-plugin)
> >
> > Micronaut is out of the scope of this voting. I will launch
> > a separate one.
> >
> > The vote is formal, see voting guidelines [2]
> >
> > +1 - to accept additional dependencies to be included to Java code
> > Guidelines [3]
> > 0 - don't care either way
> > -1 - DO NOT accept (explain why)
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> > [2] https://www.apache.org/foundation/voting.html <
> > https://www.apache.org/foundation/voting.html>
> > [3]
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Agree with Alexander. I ask the same question in the vote thread.

On Wed, May 25, 2022 at 5:47 PM Alexander Polovtcev
 wrote:
>
> Aleksandr,
>
> Can we use these annotations without the micronaut dependencies? If yes,
> why do we need micronaut at all?
>
> On Mon, May 23, 2022 at 5:34 PM Aleksandr Pakhomov  wrote:
>
> > Yes, swagger-core has its own set of annotations [1]
> >
> > [1]
> > https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview
> >
> > > On 23 May 2022, at 12:37, Alexander Polovtcev 
> > wrote:
> > >
> > > I'm not that scared of having a big library, I just believe that it might
> > > be possible to avoid using it altogether. Don't swagger generators have
> > > their own annotations?
> >
> >
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Why couldn't the handlers be annotated?

I think Kirill Gusakov (as the feature contributor) can give some
ideas about what and how should be annotated. This may require some
tweaking.

On Wed, May 25, 2022 at 6:06 PM Aleksandr Pakhomov  wrote:
>
> Hi Alexander,
>
> Yes, swagger allows to annotate classes
> and then the spec will be generated. But
> here is a question. What classes should we
> annotade?
>
> Web framework brings the convention about
> how to write an endpoint and what to annotate.
> Our own REST server implementation does not.
> It is just the number of handlers.
>
> > On 25 May 2022, at 17:46, Alexander Polovtcev  
> > wrote:
> >
> > Aleksandr,
> >
> > Can we use these annotations without the micronaut dependencies? If yes,
> > why do we need micronaut at all?
> >
> > On Mon, May 23, 2022 at 5:34 PM Aleksandr Pakhomov  wrote:
> >
> >> Yes, swagger-core has its own set of annotations [1]
> >>
> >> [1]
> >> https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview
> >>
> >>> On 23 May 2022, at 12:37, Alexander Polovtcev 
> >> wrote:
> >>>
> >>> I'm not that scared of having a big library, I just believe that it might
> >>> be possible to avoid using it altogether. Don't swagger generators have
> >>> their own annotations?
> >>
> >>
> >
>

Re: [VOTE] Add swagger dependency to Ignite 3

[Andrey Gura]

+1 from me. Open API spec could be useful in the future for
implementing external cluster management tools.

On Thu, May 26, 2022 at 11:41 AM Aleksandr Pakhomov  wrote:
>
> Hi Ivan,
>
> Dependencies that are needed for annotating
> classes are going to be included. From those
> annotations Open API spec is generated by
> maven plugin. That’s it.
>
> If you are asking about swagger ui or any
> web stuff then the answer is no. We are
> not going to include this into production.
>
> > On 26 May 2022, at 07:34, Ivan Pavlukhin  wrote:
> >
> > Hi,
> >
> > Are we going to include swagger into production packages? I always
> > thought (I might be mistaken) that swagger should be used during
> > development. Worries are usual:
> > 1. Potential vulnerabilities.
> > 2. Unintentional use of transitive dependencies.
> >
> > Best regards,
> > Ivan Pavlukhin
> >
> > чт, 26 мая 2022 г. в 00:46, Mikhail Pochatkin :
> >>
> >> +1 from me, de facto swagger standard within OpenApi.
> >>
> >> On Mon, May 23, 2022 at 7:57 PM Aleksandr Pakhomov  
> >> wrote:
> >>
> >>> Dear community,
> >>>
> >>> Discussion about 3rd party dependencies took place
> >>> and I think it is time to vote if we agreed to include
> >>> swagger dependency to the Ignite 3 or not.
> >>>
> >>> The exact list of dependencies could be fined in IEP-87 [1]
> >>> (swagger-annotations, swagger-core,
> >>> swagger-codegen-maven-plugin)
> >>>
> >>> Micronaut is out of the scope of this voting. I will launch
> >>> a separate one.
> >>>
> >>> The vote is formal, see voting guidelines [2]
> >>>
> >>> +1 - to accept additional dependencies to be included to Java code
> >>> Guidelines [3]
> >>> 0 - don't care either way
> >>> -1 - DO NOT accept (explain why)
> >>>
> >>> [1]
> >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> >>> [2] https://www.apache.org/foundation/voting.html <
> >>> https://www.apache.org/foundation/voting.html>
> >>> [3]
> >>> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Andrey Gura]

Hi,

Hmmm... It seems that [2] is just a marketing page that doesn't give
any arguments about the micronaut.

Anyway, security is a good point. Aleksandr, could you please add some
note to the IEP about security related value of the micronaut
framework?





On Thu, May 26, 2022 at 9:48 AM Mikhail Pochatkin
 wrote:
>
> Hello Igniters.
>
> I see that the main argument against Micronaut is that we need
> Micronaut+Swagger to achieve IEP 87 if we can use only Swagger. I agree
> this is a valid point and logically. But I think we can take a look into it
> from another side, from future vision. Main idea to start using micronaut
> to simplify code writing in future just because Micronaut Framework really
> helps to develop REST services in practice. For example, I think it's
> obvious that Ignite will need to implement some authentication mechanism
> and Micronaut can help to achieve it [1]. Moreover Micronaut is based on
> the same technology which is currently used in Ignite, I mean Netty. It
> means that we should speak only about vulnerabilities inside of Micronaut
> implementation, but the Micronaut project has a very big community and a
> lot of clients. In context of it I would say that bugs and vulnerabilities
> in Micronaut implementation will be found more quickly than in internal
> Ignite implementation with pure Netty. Another point about cloud native
> support [2] which also can be useful in the context of Ignite.
>
> [1] https://micronaut-projects.github.io/micronaut-security/latest/guide/
> [2] https://objectcomputing.com/expertise/cloud-engineering
>
> On Wed, May 25, 2022 at 6:45 PM Andrey Gura  wrote:
>
> > Why couldn't the handlers be annotated?
> >
> > I think Kirill Gusakov (as the feature contributor) can give some
> > ideas about what and how should be annotated. This may require some
> > tweaking.
> >
> > On Wed, May 25, 2022 at 6:06 PM Aleksandr Pakhomov 
> > wrote:
> > >
> > > Hi Alexander,
> > >
> > > Yes, swagger allows to annotate classes
> > > and then the spec will be generated. But
> > > here is a question. What classes should we
> > > annotade?
> > >
> > > Web framework brings the convention about
> > > how to write an endpoint and what to annotate.
> > > Our own REST server implementation does not.
> > > It is just the number of handlers.
> > >
> > > > On 25 May 2022, at 17:46, Alexander Polovtcev 
> > wrote:
> > > >
> > > > Aleksandr,
> > > >
> > > > Can we use these annotations without the micronaut dependencies? If
> > yes,
> > > > why do we need micronaut at all?
> > > >
> > > > On Mon, May 23, 2022 at 5:34 PM Aleksandr Pakhomov 
> > wrote:
> > > >
> > > >> Yes, swagger-core has its own set of annotations [1]
> > > >>
> > > >> [1]
> > > >>
> > https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview
> > > >>
> > > >>> On 23 May 2022, at 12:37, Alexander Polovtcev <
> > alexpolovt...@gmail.com>
> > > >> wrote:
> > > >>>
> > > >>> I'm not that scared of having a big library, I just believe that it
> > might
> > > >>> be possible to avoid using it altogether. Don't swagger generators
> > have
> > > >>> their own annotations?
> > > >>
> > > >>
> > > >
> > >
> >

Re: [VOTE] Add micronaut dependency to Ignite 3

[Andrey Gura]

Aleksandr,

please, start a new vote after updating the IEP-87. I think that
Micronaut Security is a good reason for using this framework in the
Apache Ignite.



On Wed, May 25, 2022 at 5:41 PM Andrey Gura  wrote:
>
> -1 (binding) from me.
>
> Because (if I understood correctly) the main value of the IEP-87 is
> the possibility to generate API specification and Swagger annotations
> is enough for this purpose I don't see reasons for these dependencies.
> We already have our own controllers for REST-like API's
> implementation. Why can't we just use Swagger annotations only in
> addition to our rest-api module?
>
> On Mon, May 23, 2022 at 8:08 PM Aleksandr Pakhomov  wrote:
> >
> > Dear community,
> >
> > Micronaut-based REST server implementation was a hot
> > topic we discussed in the previous week. So, I've separeted
> > votes about swagger and micronaut. This vote is about
> > adding micronaut to the Ignite 3.
> >
> > The exact list of dependencies could be fined in IEP-87 [1]
> > io.micronaut.serde:micronaut-serde
> > io.micronaut:micronaut-context
> > io.micronaut:micronaut-http
> > io.micronaut:micronaut-inject
> > io.micronaut:micronaut-http-server
> > io.micronaut:micronaut-runtime
> > io.micronaut:micronaut-core
> > io.micronaut:micronaut-http-server-netty
> > io.micronaut:micronaut-http-netty
> > io.micronaut:micronaut-buffer-netty
> > io.micronaut:micronaut-aop
> > io.micronaut:micronaut-core-reactive
> > Io.micronaut:micronaut-json-core
> > io.micronaut:micronaut-jackson-core
> >
> > Swagger is out of the scope of this voting.
> >
> > The vote is formal, see voting guidelines [2]
> >
> > +1 - to accept additional dependencies to be included to Java code 
> > Guidelines [3]
> > 0 - don't care either way
> > -1 - DO NOT accept (explain why)
> >
> > This vote will be open for at least 4 days till Fri May 27, 2022,
> > 21:00 Moscow TZ.
> >
> > [1] 
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> > [2] https://www.apache.org/foundation/voting.html 
> > <https://www.apache.org/foundation/voting.html>
> > [3] 
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> >  
> > <https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries>
> > [4] 
> > https://www.timeanddate.com/countdown/generic?iso=20220527T21&p0=166&msg=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3&font=cursive

Re: [VOTE] Add micronaut dependency to Ignite 3 (reopened)

[Andrey Gura]

+1

On Mon, May 30, 2022 at 12:13 PM Alexander Polovtcev
 wrote:
>
> Aleksandr,
> Thanks for the update, but can you please explain what security
> capabilities are planned to be used? Maybe it should be included in the IEP
> as well.
>
> On Sun, May 29, 2022 at 11:03 PM Aleksandr Pakhomov 
> wrote:
>
> > Dear community,
> >
> > We’ve had a productive discussion about
> > a micronaut dependency [1]. As a result,
> > we’ve came to the conclusion that the
> > security support in micronaut is a good
> > reason to use this library. I am reopening the
> > vote for adding the micronaut dependency to
> > the rest module. Also, micronaut-serde is
> > dropped from the list because Ignite 3
> > already uses jackson as a serialisation lib.
> >
> > List of dependencies:
> > io.micronaut:micronaut-inject:jar:3.4.1
> > io.micronaut:micronaut-core:jar:3.4.1
> > io.micronaut:micronaut-http-server-netty:jar:3.4.1
> > io.micronaut:micronaut-http-server:jar:3.4.1
> > io.micronaut:micronaut-router:jar:3.4.1
> > io.micronaut:micronaut-http-netty:jar:3.4.1
> > io.micronaut:micronaut-buffer-netty:jar:3.4.1
> > io.micronaut:micronaut-runtime:jar:3.4.1
> > io.micronaut:micronaut-http:jar:3.4.1
> > io.micronaut:micronaut-aop:jar:3.4.1
> > io.micronaut:micronaut-context:jar:3.4.1
> > io.micronaut:micronaut-core-reactive:jar:3.4.1
> >
> > More information about motivation and
> > implementation details could be found
> > in IEP-87 [2].
> >
> > The vote is formal, see voting guidelines [3].
> >
> > +1 - to accept additional dependencies to be included to Java code
> > Guidelines [4]
> > 0 - don’t care either way
> > -1 - DO NOT accept (explain why)
> >
> > This vote will be open for at leas 3 days till Wed June 1, 2022,
> > 23:00 Moscow TZ [5].
> >
> >
> > [1] https://lists.apache.org/thread/0nq6wx8t9r036mrjsk6n592gwnvvqbhs <
> > https://lists.apache.org/thread/0nq6wx8t9r036mrjsk6n592gwnvvqbhs>
> > [2]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST
> > <
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-87:+Open+API+support+for+REST>
> >
> > [3] https://www.apache.org/foundation/voting.html <
> > https://www.apache.org/foundation/voting.html>
> > [4]
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> > <
> > https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
> > >
> > [5]
> > https://www.timeanddate.com/countdown/generic?iso=20220601T23&p0=166&msg=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3+%28reopened%29&font=cursive
> > <
> > https://www.timeanddate.com/countdown/generic?iso=20220601T23&p0=166&msg=[VOTE]+Add+micronaut+dependency+to+Ignite+3+(reopened)&font=cursive>
> >
>
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [RESULT][VOTE] Add swagger dependency to Ignite 3

[Andrey Gura]

Aleksandr,

could you please update
https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries

On Mon, May 30, 2022 at 12:33 PM Aleksandr Pakhomov  wrote:
>
> Hello Igniters,
>
> The swagger dependency has been accepted.
>
> 3 - “+1” votes received.
> 1 - “0.5” vote received.
>
> Here are +1 votes received:
>
> - Alexander Polovtcev
> - Andrey Gura
> - Ivan Pavlukhin
>
> Here is -0.5 vote received:
>
> - Ilya Kasnacheev (binding)
>
> Link to the voting thread -
> https://lists.apache.org/thread/dntnxcyzpddjg5yc5vkd8lxjtd5z54rc 
> <https://lists.apache.org/thread/dntnxcyzpddjg5yc5vkd8lxjtd5z54rc>
>
> Best regards,
> Aleksandr

[ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igniters,

our release schedule has shifted a bit. But it is time for a code
freeze and a new branch creation.

The following issues is still in progress (not an issues status, but
work state):

Data rebalancing
https://issues.apache.org/jira/browse/IGNITE-14209

CLI MVP
https://issues.apache.org/jira/browse/IGNITE-16971

[Native Persistence 3.0] End-to-end test for persistent PageMemory
https://issues.apache.org/jira/browse/IGNITE-17107

SQL API: Implement query metadata
https://issues.apache.org/jira/browse/IGNITE-16962

SQL API: Add batched DML queries support.
https://issues.apache.org/jira/browse/IGNITE-16963

SQL API: Examples.
https://issues.apache.org/jira/browse/IGNITE-17088

Please, give some planned timelines for these issues. I would like to
create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
Otherwise, extra steps will be required to include the PR's to the
release branch.

Thanks!

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igniters,

ignite-3.0.0-alpha5 release branch has been created. But the following
issues are still in progress:

Data rebalancing
https://issues.apache.org/jira/browse/IGNITE-14209

SQL API: Add batched DML queries support.
https://issues.apache.org/jira/browse/IGNITE-16963

SQL API: Examples.
https://issues.apache.org/jira/browse/IGNITE-17088

Please, make sure these commits will be merged to the main and to the
release branch.

Thanks!

On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov  wrote:
>
> Hi Andrey,
>
> As for CLI MVP, the planned timeline is today till 21:00.
> Probably, will be ready in an hour. Just waiting for CI build.
>
> Best regards,
> Aleksandr
>
> > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> >
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
>

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

And, of course, there is a code freeze for the release branch. So,
permission is required to make any changes to the release branch.
Don't hesitate to discuss such changes in this topic (except three
issues from the previous post).

On Mon, Jun 6, 2022 at 10:52 PM Andrey Gura  wrote:
>
> Igniters,
>
> ignite-3.0.0-alpha5 release branch has been created. But the following
> issues are still in progress:
>
> Data rebalancing
> https://issues.apache.org/jira/browse/IGNITE-14209
>
> SQL API: Add batched DML queries support.
> https://issues.apache.org/jira/browse/IGNITE-16963
>
> SQL API: Examples.
> https://issues.apache.org/jira/browse/IGNITE-17088
>
> Please, make sure these commits will be merged to the main and to the
> release branch.
>
> Thanks!
>
> On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov  wrote:
> >
> > Hi Andrey,
> >
> > As for CLI MVP, the planned timeline is today till 21:00.
> > Probably, will be ready in an hour. Just waiting for CI build.
> >
> > Best regards,
> > Aleksandr
> >
> > > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> > >
> > > Igniters,
> > >
> > > our release schedule has shifted a bit. But it is time for a code
> > > freeze and a new branch creation.
> > >
> > > The following issues is still in progress (not an issues status, but
> > > work state):
> > >
> > > Data rebalancing
> > > https://issues.apache.org/jira/browse/IGNITE-14209
> > >
> > > CLI MVP
> > > https://issues.apache.org/jira/browse/IGNITE-16971
> > >
> > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > https://issues.apache.org/jira/browse/IGNITE-17107
> > >
> > > SQL API: Implement query metadata
> > > https://issues.apache.org/jira/browse/IGNITE-16962
> > >
> > > SQL API: Add batched DML queries support.
> > > https://issues.apache.org/jira/browse/IGNITE-16963
> > >
> > > SQL API: Examples.
> > > https://issues.apache.org/jira/browse/IGNITE-17088
> > >
> > > Please, give some planned timelines for these issues. I would like to
> > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > Otherwise, extra steps will be required to include the PR's to the
> > > release branch.
> > >
> > > Thanks!
> >

Re: Request for contributors permissions

[Andrey Gura]

Vadim,

Contributor role has been granted.

On Mon, Jun 6, 2022 at 11:36 AM Vadim Pakhnushev
 wrote:
>
> Hi!
>
> Could you please grant contributor access to JIRA and Confluence?
> My username is vpakhnushev.
>
> Thanks!

Re: Request for contributors permissions

[Andrey Gura]

Aleksey,

contributor role has been granted.

On Mon, Jun 6, 2022 at 1:56 PM Aleksey Demakov  wrote:
>
> Hi Ignite community,
>
> My name is Aleksey Demakov, I'd like to contribute to the Apache
> Ignite project. Could you please grant me contributor permissions to
> jira and wiki. My username is ademakov.
>
> Thanks,
> Aleksey

Re: Re[2]: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Thanks, Igor.

These issues are included in the release. Please, move forward and let
me know when PR's will be ready to be merged (all checks done, review
done, tickets have corresponding comments from reviewrs).

On Tue, Jun 7, 2022 at 2:19 PM Игорь Гусев  wrote:
>
>
> Hi, I did some PRs with docs for new features. Can you include them in 
> release?
>
> https://github.com/apache/ignite-3/pull/850
> https://github.com/apache/ignite-3/pull/856
> https://github.com/apache/ignite-3/pull/857
>
>
> >Вторник, 7 июня 2022, 0:09 +04:00 от Andrey Gura :
> >
> >And, of course, there is a code freeze for the release branch. So,
> >permission is required to make any changes to the release branch.
> >Don't hesitate to discuss such changes in this topic (except three
> >issues from the previous post).
> >
> >On Mon, Jun 6, 2022 at 10:52 PM Andrey Gura < ag...@apache.org > wrote:
> >>
> >> Igniters,
> >>
> >> ignite-3.0.0-alpha5 release branch has been created. But the following
> >> issues are still in progress:
> >>
> >> Data rebalancing
> >>  https://issues.apache.org/jira/browse/IGNITE-14209
> >>
> >> SQL API: Add batched DML queries support.
> >>  https://issues.apache.org/jira/browse/IGNITE-16963
> >>
> >> SQL API: Examples.
> >>  https://issues.apache.org/jira/browse/IGNITE-17088
> >>
> >> Please, make sure these commits will be merged to the main and to the
> >> release branch.
> >>
> >> Thanks!
> >>
> >> On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov < apk...@gmail.com > 
> >> wrote:
> >> >
> >> > Hi Andrey,
> >> >
> >> > As for CLI MVP, the planned timeline is today till 21:00.
> >> > Probably, will be ready in an hour. Just waiting for CI build.
> >> >
> >> > Best regards,
> >> > Aleksandr
> >> >
> >> > > On 6 Jun 2022, at 17:56, Andrey Gura < ag...@apache.org > wrote:
> >> > >
> >> > > Igniters,
> >> > >
> >> > > our release schedule has shifted a bit. But it is time for a code
> >> > > freeze and a new branch creation.
> >> > >
> >> > > The following issues is still in progress (not an issues status, but
> >> > > work state):
> >> > >
> >> > > Data rebalancing
> >> > >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> > >
> >> > > CLI MVP
> >> > >  https://issues.apache.org/jira/browse/IGNITE-16971
> >> > >
> >> > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> >> > >  https://issues.apache.org/jira/browse/IGNITE-17107
> >> > >
> >> > > SQL API: Implement query metadata
> >> > >  https://issues.apache.org/jira/browse/IGNITE-16962
> >> > >
> >> > > SQL API: Add batched DML queries support.
> >> > >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> > >
> >> > > SQL API: Examples.
> >> > >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> > >
> >> > > Please, give some planned timelines for these issues. I would like to
> >> > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> >> > > Otherwise, extra steps will be required to include the PR's to the
> >> > > release branch.
> >> > >
> >> > > Thanks!
> >> >
>
>
> --
> Igor Gusev
>

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Great! Thanks, Slava!

On Tue, Jun 7, 2022 at 2:50 PM Вячеслав Коптилин
 wrote:
>
> Hello Andrey,
>
> Data rebalancing (https://issues.apache.org/jira/browse/IGNITE-14209) is
> already done and merged to main & ignite-3.0.0-alpha5 branches.
>
> Thanks,
> S.
>
>
> пн, 6 июн. 2022 г. в 22:52, Andrey Gura :
>
> > Igniters,
> >
> > ignite-3.0.0-alpha5 release branch has been created. But the following
> > issues are still in progress:
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, make sure these commits will be merged to the main and to the
> > release branch.
> >
> > Thanks!
> >
> > On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov 
> > wrote:
> > >
> > > Hi Andrey,
> > >
> > > As for CLI MVP, the planned timeline is today till 21:00.
> > > Probably, will be ready in an hour. Just waiting for CI build.
> > >
> > > Best regards,
> > > Aleksandr
> > >
> > > > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> > > >
> > > > Igniters,
> > > >
> > > > our release schedule has shifted a bit. But it is time for a code
> > > > freeze and a new branch creation.
> > > >
> > > > The following issues is still in progress (not an issues status, but
> > > > work state):
> > > >
> > > > Data rebalancing
> > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > >
> > > > CLI MVP
> > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > >
> > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > >
> > > > SQL API: Implement query metadata
> > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > >
> > > > SQL API: Add batched DML queries support.
> > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > >
> > > > SQL API: Examples.
> > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > >
> > > > Please, give some planned timelines for these issues. I would like to
> > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > Otherwise, extra steps will be required to include the PR's to the
> > > > release branch.
> > > >
> > > > Thanks!
> > >
> >

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Andrey,

while IGNITE-17128 is okay just because it demonstrates some
functionality, IGNITE-17129 doesn't look critical and could be fixed
in some future release. WDYT?

On Tue, Jun 7, 2022 at 5:58 PM Андрей Хитрин  wrote:
>
> Hello! Could you please add the following issues to the release?
>
> * https://issues.apache.org/jira/browse/IGNITE-17128:
> PersistentPageMemoryStorageExample cannot be run twice
> * https://issues.apache.org/jira/browse/IGNITE-17129: cli tool doesn't
> expand tilde in a config path
>
> пн, 6 июн. 2022 г. в 19:56, Andrey Gura :
>
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
> >
>
>
> --
> Андрей Хитрин

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Kirill, thanks for help!

I've already targeted this issue to the current release. Make me know
when the issue will be ready to merge.

On Tue, Jun 7, 2022 at 5:59 PM ткаленко кирилл  wrote:
>
> Hello everyone, I would like to fix the identified bug when destroying the 
> table for persistent PageMemory, I could fix it tomorrow, can we add it to 
> the scope?
>
> https://issues.apache.org/jira/browse/IGNITE-17128

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Aleksandr,

okay, looks like a harmless change with documentation fixes. Targeted
to the release. Notify me when it will be ready for merge.

On Tue, Jun 7, 2022 at 6:04 PM Aleksandr Pakhomov  wrote:
>
> Hi Andrey,
>
> Could you please include this ticket into release?
> https://issues.apache.org/jira/browse/IGNITE-17126 
> <https://issues.apache.org/jira/browse/IGNITE-17126>
>
> Thanks
>
> > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> >
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
>

Re: Re[2]: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igor,

Documentation issues have been targeted to the current release. I'm
waiting for "ready for merge" notification.

Thanks!

On Tue, Jun 7, 2022 at 6:20 PM Игорь Гусев  wrote:
>
>
> Please include these 2 issues into release:
> https://issues.apache.org/jira/browse/IGNITE-17127
> https://issues.apache.org/jira/browse/IGNITE-17106
>
>
> >Вторник, 7 июня 2022, 19:07 +04:00 от Andrey Gura :
> >
> >Aleksandr,
> >
> >okay, looks like a harmless change with documentation fixes. Targeted
> >to the release. Notify me when it will be ready for merge.
> >
> >On Tue, Jun 7, 2022 at 6:04 PM Aleksandr Pakhomov < apk...@gmail.com > wrote:
> >>
> >> Hi Andrey,
> >>
> >> Could you please include this ticket into release?
> >>  https://issues.apache.org/jira/browse/IGNITE-17126 < 
> >> https://issues.apache.org/jira/browse/IGNITE-17126 >
> >>
> >> Thanks
> >>
> >> > On 6 Jun 2022, at 17:56, Andrey Gura < ag...@apache.org > wrote:
> >> >
> >> > Igniters,
> >> >
> >> > our release schedule has shifted a bit. But it is time for a code
> >> > freeze and a new branch creation.
> >> >
> >> > The following issues is still in progress (not an issues status, but
> >> > work state):
> >> >
> >> > Data rebalancing
> >> >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> >
> >> > CLI MVP
> >> >  https://issues.apache.org/jira/browse/IGNITE-16971
> >> >
> >> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> >> >  https://issues.apache.org/jira/browse/IGNITE-17107
> >> >
> >> > SQL API: Implement query metadata
> >> >  https://issues.apache.org/jira/browse/IGNITE-16962
> >> >
> >> > SQL API: Add batched DML queries support.
> >> >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> >
> >> > SQL API: Examples.
> >> >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> >
> >> > Please, give some planned timelines for these issues. I would like to
> >> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> >> > Otherwise, extra steps will be required to include the PR's to the
> >> > release branch.
> >> >
> >> > Thanks!
> >>
>
>
> --
> Igor Gusev
>

Re: Re[2]: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Igor,

Thanks for your efforts. All changes have been merged.

On Tue, Jun 7, 2022 at 5:41 PM Игорь Гусев  wrote:
>
>
>
> I fixed issues with the PRs mentioned earlier, so hopefully they can be 
> merged.
>
> >Вторник, 7 июня 2022, 16:45 +04:00 от Andrey Gura :
> >
> >Great! Thanks, Slava!
> >
> >On Tue, Jun 7, 2022 at 2:50 PM Вячеслав Коптилин
> >< slava.kopti...@gmail.com > wrote:
> >>
> >> Hello Andrey,
> >>
> >> Data rebalancing (https://issues.apache.org/jira/browse/IGNITE-14209) is
> >> already done and merged to main & ignite-3.0.0-alpha5 branches.
> >>
> >> Thanks,
> >> S.
> >>
> >>
> >> пн, 6 июн. 2022 г. в 22:52, Andrey Gura < ag...@apache.org >:
> >>
> >> > Igniters,
> >> >
> >> > ignite-3.0.0-alpha5 release branch has been created. But the following
> >> > issues are still in progress:
> >> >
> >> > Data rebalancing
> >> >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> >
> >> > SQL API: Add batched DML queries support.
> >> >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> >
> >> > SQL API: Examples.
> >> >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> >
> >> > Please, make sure these commits will be merged to the main and to the
> >> > release branch.
> >> >
> >> > Thanks!
> >> >
> >> > On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov < apk...@gmail.com >
> >> > wrote:
> >> > >
> >> > > Hi Andrey,
> >> > >
> >> > > As for CLI MVP, the planned timeline is today till 21:00.
> >> > > Probably, will be ready in an hour. Just waiting for CI build.
> >> > >
> >> > > Best regards,
> >> > > Aleksandr
> >> > >
> >> > > > On 6 Jun 2022, at 17:56, Andrey Gura < ag...@apache.org > wrote:
> >> > > >
> >> > > > Igniters,
> >> > > >
> >> > > > our release schedule has shifted a bit. But it is time for a code
> >> > > > freeze and a new branch creation.
> >> > > >
> >> > > > The following issues is still in progress (not an issues status, but
> >> > > > work state):
> >> > > >
> >> > > > Data rebalancing
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-14209
> >> > > >
> >> > > > CLI MVP
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-16971
> >> > > >
> >> > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-17107
> >> > > >
> >> > > > SQL API: Implement query metadata
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-16962
> >> > > >
> >> > > > SQL API: Add batched DML queries support.
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-16963
> >> > > >
> >> > > > SQL API: Examples.
> >> > > >  https://issues.apache.org/jira/browse/IGNITE-17088
> >> > > >
> >> > > > Please, give some planned timelines for these issues. I would like to
> >> > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> >> > > > Otherwise, extra steps will be required to include the PR's to the
> >> > > > release branch.
> >> > > >
> >> > > > Thanks!
> >> > >
> >> >
>
>
> --
> Igor Gusev
>

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

On Wed, Jun 8, 2022 at 11:13 AM Pavel Tupitsyn  wrote:
>
> Igniters, I'd like to include the following ticket as well:
> IGNITE-17057 Thin 3.0: Implement synchronous SQL API for java thin client
> [1]
>
> Would be great to have the same API available in server and client modes.
> It'll be in main in a couple hours.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-17057
>
> On Wed, Jun 8, 2022 at 9:27 AM Андрей Хитрин 
> wrote:
>
> > Looks reasonable to me. IGNITE-17129 could wait for the next release. Thank
> > you!
> >
> > вт, 7 июн. 2022 г. в 20:05, Andrey Gura :
> >
> > > Andrey,
> > >
> > > while IGNITE-17128 is okay just because it demonstrates some
> > > functionality, IGNITE-17129 doesn't look critical and could be fixed
> > > in some future release. WDYT?
> > >
> > > On Tue, Jun 7, 2022 at 5:58 PM Андрей Хитрин 
> > > wrote:
> > > >
> > > > Hello! Could you please add the following issues to the release?
> > > >
> > > > * https://issues.apache.org/jira/browse/IGNITE-17128:
> > > > PersistentPageMemoryStorageExample cannot be run twice
> > > > * https://issues.apache.org/jira/browse/IGNITE-17129: cli tool doesn't
> > > > expand tilde in a config path
> > > >
> > > > пн, 6 июн. 2022 г. в 19:56, Andrey Gura :
> > > >
> > > > > Igniters,
> > > > >
> > > > > our release schedule has shifted a bit. But it is time for a code
> > > > > freeze and a new branch creation.
> > > > >
> > > > > The following issues is still in progress (not an issues status, but
> > > > > work state):
> > > > >
> > > > > Data rebalancing
> > > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > > >
> > > > > CLI MVP
> > > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > > >
> > > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > > >
> > > > > SQL API: Implement query metadata
> > > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > > >
> > > > > SQL API: Add batched DML queries support.
> > > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > > >
> > > > > SQL API: Examples.
> > > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > > >
> > > > > Please, give some planned timelines for these issues. I would like to
> > > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > > Otherwise, extra steps will be required to include the PR's to the
> > > > > release branch.
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > Андрей Хитрин
> > >
> >
> >
> > --
> > Андрей Хитрин
> >

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Pavel,

The issue is targeted to the alpha5 release.

On Wed, Jun 8, 2022 at 11:13 AM Pavel Tupitsyn  wrote:
>
> Igniters, I'd like to include the following ticket as well:
> IGNITE-17057 Thin 3.0: Implement synchronous SQL API for java thin client
> [1]
>
> Would be great to have the same API available in server and client modes.
> It'll be in main in a couple hours.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-17057
>
> On Wed, Jun 8, 2022 at 9:27 AM Андрей Хитрин 
> wrote:
>
> > Looks reasonable to me. IGNITE-17129 could wait for the next release. Thank
> > you!
> >
> > вт, 7 июн. 2022 г. в 20:05, Andrey Gura :
> >
> > > Andrey,
> > >
> > > while IGNITE-17128 is okay just because it demonstrates some
> > > functionality, IGNITE-17129 doesn't look critical and could be fixed
> > > in some future release. WDYT?
> > >
> > > On Tue, Jun 7, 2022 at 5:58 PM Андрей Хитрин 
> > > wrote:
> > > >
> > > > Hello! Could you please add the following issues to the release?
> > > >
> > > > * https://issues.apache.org/jira/browse/IGNITE-17128:
> > > > PersistentPageMemoryStorageExample cannot be run twice
> > > > * https://issues.apache.org/jira/browse/IGNITE-17129: cli tool doesn't
> > > > expand tilde in a config path
> > > >
> > > > пн, 6 июн. 2022 г. в 19:56, Andrey Gura :
> > > >
> > > > > Igniters,
> > > > >
> > > > > our release schedule has shifted a bit. But it is time for a code
> > > > > freeze and a new branch creation.
> > > > >
> > > > > The following issues is still in progress (not an issues status, but
> > > > > work state):
> > > > >
> > > > > Data rebalancing
> > > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > > >
> > > > > CLI MVP
> > > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > > >
> > > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > > >
> > > > > SQL API: Implement query metadata
> > > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > > >
> > > > > SQL API: Add batched DML queries support.
> > > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > > >
> > > > > SQL API: Examples.
> > > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > > >
> > > > > Please, give some planned timelines for these issues. I would like to
> > > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > > Otherwise, extra steps will be required to include the PR's to the
> > > > > release branch.
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> > > >
> > > > --
> > > > Андрей Хитрин
> > >
> >
> >
> > --
> > Андрей Хитрин
> >

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Thanks, Yuri!

On Wed, Jun 8, 2022 at 1:48 PM Юрий  wrote:
>
> Seems  SQL API: Add batched DML queries support [1] will be moved to next
> alpha, so pleas don't waiting for it.
>
> In the same time SQL API: Examples [2] already done and merged to alpha 5
> branch.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-16963
> [2] https://issues.apache.org/jira/browse/IGNITE-17088
>
> пн, 6 июн. 2022 г. в 22:52, Andrey Gura :
>
> > Igniters,
> >
> > ignite-3.0.0-alpha5 release branch has been created. But the following
> > issues are still in progress:
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, make sure these commits will be merged to the main and to the
> > release branch.
> >
> > Thanks!
> >
> > On Mon, Jun 6, 2022 at 6:02 PM Aleksandr Pakhomov 
> > wrote:
> > >
> > > Hi Andrey,
> > >
> > > As for CLI MVP, the planned timeline is today till 21:00.
> > > Probably, will be ready in an hour. Just waiting for CI build.
> > >
> > > Best regards,
> > > Aleksandr
> > >
> > > > On 6 Jun 2022, at 17:56, Andrey Gura  wrote:
> > > >
> > > > Igniters,
> > > >
> > > > our release schedule has shifted a bit. But it is time for a code
> > > > freeze and a new branch creation.
> > > >
> > > > The following issues is still in progress (not an issues status, but
> > > > work state):
> > > >
> > > > Data rebalancing
> > > > https://issues.apache.org/jira/browse/IGNITE-14209
> > > >
> > > > CLI MVP
> > > > https://issues.apache.org/jira/browse/IGNITE-16971
> > > >
> > > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > > https://issues.apache.org/jira/browse/IGNITE-17107
> > > >
> > > > SQL API: Implement query metadata
> > > > https://issues.apache.org/jira/browse/IGNITE-16962
> > > >
> > > > SQL API: Add batched DML queries support.
> > > > https://issues.apache.org/jira/browse/IGNITE-16963
> > > >
> > > > SQL API: Examples.
> > > > https://issues.apache.org/jira/browse/IGNITE-17088
> > > >
> > > > Please, give some planned timelines for these issues. I would like to
> > > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > > Otherwise, extra steps will be required to include the PR's to the
> > > > release branch.
> > > >
> > > > Thanks!
> > >
> >
>
>
> --
> Живи с улыбкой! :D

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Alexander,

It looks like a blocker. The issue has been targeted to alpha5 release.

On Wed, Jun 8, 2022 at 7:30 PM Alexander Polovtcev
 wrote:
>
> Hello, dear Igniters.
> I've discovered a bug regarding the Pluggable Storage Engines feature.
> Please consider adding it to the release scope:
> https://issues.apache.org/jira/browse/IGNITE-17139
>
> On Mon, Jun 6, 2022 at 5:56 PM Andrey Gura  wrote:
>
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Targeted to the release.

Thanks, Alexander.

On Thu, Jun 9, 2022 at 2:09 PM Alexander Polovtcev
 wrote:
>
> Hello again, there's another issue that blocks building the release branch:
> https://issues.apache.org/jira/browse/IGNITE-17144. Fix will be provided
> shortly
>
> On Mon, Jun 6, 2022 at 5:56 PM Andrey Gura  wrote:
>
> > Igniters,
> >
> > our release schedule has shifted a bit. But it is time for a code
> > freeze and a new branch creation.
> >
> > The following issues is still in progress (not an issues status, but
> > work state):
> >
> > Data rebalancing
> > https://issues.apache.org/jira/browse/IGNITE-14209
> >
> > CLI MVP
> > https://issues.apache.org/jira/browse/IGNITE-16971
> >
> > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > https://issues.apache.org/jira/browse/IGNITE-17107
> >
> > SQL API: Implement query metadata
> > https://issues.apache.org/jira/browse/IGNITE-16962
> >
> > SQL API: Add batched DML queries support.
> > https://issues.apache.org/jira/browse/IGNITE-16963
> >
> > SQL API: Examples.
> > https://issues.apache.org/jira/browse/IGNITE-17088
> >
> > Please, give some planned timelines for these issues. I would like to
> > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > Otherwise, extra steps will be required to include the PR's to the
> > release branch.
> >
> > Thanks!
> >
>
>
> --
> With regards,
> Aleksandr Polovtcev

Re: [ANNOUNCE] Apache Ignite 3.0.0 alpha 5: Code freeze

[Andrey Gura]

Thanks a lot to all! We are ready to vote.

On Thu, Jun 9, 2022 at 2:28 PM Andrey Gura  wrote:
>
> Targeted to the release.
>
> Thanks, Alexander.
>
> On Thu, Jun 9, 2022 at 2:09 PM Alexander Polovtcev
>  wrote:
> >
> > Hello again, there's another issue that blocks building the release branch:
> > https://issues.apache.org/jira/browse/IGNITE-17144. Fix will be provided
> > shortly
> >
> > On Mon, Jun 6, 2022 at 5:56 PM Andrey Gura  wrote:
> >
> > > Igniters,
> > >
> > > our release schedule has shifted a bit. But it is time for a code
> > > freeze and a new branch creation.
> > >
> > > The following issues is still in progress (not an issues status, but
> > > work state):
> > >
> > > Data rebalancing
> > > https://issues.apache.org/jira/browse/IGNITE-14209
> > >
> > > CLI MVP
> > > https://issues.apache.org/jira/browse/IGNITE-16971
> > >
> > > [Native Persistence 3.0] End-to-end test for persistent PageMemory
> > > https://issues.apache.org/jira/browse/IGNITE-17107
> > >
> > > SQL API: Implement query metadata
> > > https://issues.apache.org/jira/browse/IGNITE-16962
> > >
> > > SQL API: Add batched DML queries support.
> > > https://issues.apache.org/jira/browse/IGNITE-16963
> > >
> > > SQL API: Examples.
> > > https://issues.apache.org/jira/browse/IGNITE-17088
> > >
> > > Please, give some planned timelines for these issues. I would like to
> > > create the ignite-3.0.0-alpha5 branch today and announce Code Freeze.
> > > Otherwise, extra steps will be required to include the PR's to the
> > > release branch.
> > >
> > > Thanks!
> > >
> >
> >
> > --
> > With regards,
> > Aleksandr Polovtcev

[VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Andrey Gura]

Dear Community,

In the last few month the following major features have been added:

  - Pluggable storages: ability to choose a specific storage for a
table (RocksDB based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improved UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API for executing SQL
queries on a cluster.

This is an important and interesting milestone and we should share the
current state
with the broader community. I propose to release 3.0.0 Alpha5 with the
features listed above.

Please vote.

---

Release candidate:
https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha5-rc1/
Maven staging:
https://repository.apache.org/content/repositories/orgapacheignite-1551/
Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha5-rc1

+1 - accept Apache Ignite 3.0.0-alpha5 RC1
 0 - don't care either way
-1 - DO NOT accept Apache Ignite 3.0.0-alpha5 RC1 (explain why)

Voting guidelines: https://www.apache.org/foundation/voting.html
How to verify the release: https://www.apache.org/info/verification.html

The vote will be open for 96 hours and will close on June 13th at 6:45pm
https://www.timeanddate.com/countdown/to?iso=20220613T1845&p0=352

Re: [VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Andrey Gura]

+1 (binding)

Signatures - checked.
Examples - checked.
Build - checked.

On Tue, Jun 14, 2022 at 11:56 AM Pavel Tupitsyn  wrote:
>
> +1 (binding)
>
> On Fri, Jun 10, 2022 at 4:35 PM Igor Sapego  wrote:
>
> > +1
> >
> > Best Regards,
> > Igor
> >
> >
> > On Fri, Jun 10, 2022 at 1:37 AM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > +1
> > >
> > > On Thu, Jun 9, 2022 at 9:17 AM Alexander Polovtcev <
> > > alexpolovt...@gmail.com>
> > > wrote:
> > >
> > > > Looks good, so many great features! +1
> > > >
> > > > On Thu, Jun 9, 2022 at 6:42 PM Andrey Gura  wrote:
> > > >
> > > > > Dear Community,
> > > > >
> > > > > In the last few month the following major features have been added:
> > > > >
> > > > >   - Pluggable storages: ability to choose a specific storage for a
> > > > > table (RocksDB based storage, Page memory persistent and in-memory
> > > > > storage) with some known limitations.
> > > > >   - Compute API (A simple remote job execution): The first phase of
> > > > > Compute API design and implementation. Of course, with known
> > > > > limitations.
> > > > >   - Data colocation: The colocation key concept replaces the affinity
> > > > > key concept. DDL introduces COLOCATE BY clause. Colocated job
> > > > > execution.
> > > > >   - Open API for the Ignite REST endpoints: A Specification to
> > > > > generate a client for any language + auto-generated docs for REST
> > API.
> > > > >   - Ignite REPL: The Ignite CLI as a REPL with autocompletion and
> > > > improved
> > > > > UX.
> > > > >   - Cluster lifecycle: It introduces cluster initialization logic and
> > > > > allows to specify cluster management and meta storage groups.
> > Improved
> > > > > node join protocol.
> > > > >   - Local and distributed recovery: Now it is possible to restart a
> > > > > cluster/node without data loss.
> > > > >   - Data rebalance improvements (in progress and could be excluded
> > > > > from the release), including dynamically changing the number of
> > > > > partition replicas.
> > > > >   - Robust client connection with seamless reconnection support and
> > > > > retry policies.
> > > > >   - Java API for SQL: A simplified API for executing SQL
> > > > > queries on a cluster.
> > > > >
> > > > > This is an important and interesting milestone and we should share
> > the
> > > > > current state
> > > > > with the broader community. I propose to release 3.0.0 Alpha5 with
> > the
> > > > > features listed above.
> > > > >
> > > > > Please vote.
> > > > >
> > > > > ---
> > > > >
> > > > > Release candidate:
> > > > > https://dist.apache.org/repos/dist/dev/ignite/3.0.0-alpha5-rc1/
> > > > > Maven staging:
> > > > >
> > > https://repository.apache.org/content/repositories/orgapacheignite-1551/
> > > > > Git tag: https://github.com/apache/ignite-3/tree/3.0.0-alpha5-rc1
> > > > >
> > > > > +1 - accept Apache Ignite 3.0.0-alpha5 RC1
> > > > >  0 - don't care either way
> > > > > -1 - DO NOT accept Apache Ignite 3.0.0-alpha5 RC1 (explain why)
> > > > >
> > > > > Voting guidelines: https://www.apache.org/foundation/voting.html
> > > > > How to verify the release:
> > > https://www.apache.org/info/verification.html
> > > > >
> > > > > The vote will be open for 96 hours and will close on June 13th at
> > > 6:45pm
> > > > > https://www.timeanddate.com/countdown/to?iso=20220613T1845&p0=352
> > > > >
> > > >
> > > >
> > > > --
> > > > With regards,
> > > > Aleksandr Polovtcev
> > > >
> > >
> >

[RESULT][VOTE] Release Apache Ignite 3.0.0-alpha5 RC1

[Andrey Gura]

Igniters,

Apache Ignite 3.0.0-alpha5 RC1 has been accepted.

7 "+1" votes received:
- Semyon Danilov
- Kirill Tkalenko
- Alexander Polovtcev
- Valentin Kulichenko (binding)
- Igor Sapego (binding)
- Pavel Tupitsyn (binding)
    - Andrey Gura (binding)


No "0" or "-1" votes.

Vote thread:
https://lists.apache.org/thread/df97213fjw4t0hhsn5lz9ktvc6ropxyk

[ANNOUNCE] Apache Ignite 3.0.0-alpha5 is released!

[Andrey Gura]

Igniters,

I'm happy to announce that the 5th alpha version of Ignite 3 is out!

On top of the functionality that was previously released, Alpha 5 adds
the following major features:

 - Pluggable storages: ability to choose a specific storage for a
table (RocksDB based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improved UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API for executing SQL
queries on a cluster.

Code examples have been added for the new features, you can check them out
here: https://github.com/apache/ignite-3/tree/3.0.0-alpha5/examples

The best way to try the release out is to go through the Getting Started
Guide: https://ignite.apache.org/docs/3.0.0-alpha

If there are any questions, issues, or thoughts, please do not hesitate to
reply to this email. Ignite Community is welcoming any feedback and will
consider it in future development.

Re: Questions about SQL transactions

[Andrey Gura]

Hi,

> 1. Is MVCC the only atomicity mode that supports SQL transactions?

Yes, only TRANSACTIONAL_SNAPSHOT atomicity mode is applicable to SQL.

> What if
I create a table with TRANSACTIONAL atomicity? Will it fall back to like
optimistic/pessimistic transactions (OCC/PCC), or just no transaction
supports at all? If OCC/PCC is applied, what are the limitations?

TRANSACTIONAL atomicity mode works only with KV API.

> 2 (1-yes). Is that possible to implement an external cache store that
provides some sort of transaction support that could work for SQL OCC/PCC,
any suggestions on supplemental materials that are helpful for the goal?

No, it is impossible now. Moreover, SQL is not intended for queries
over cache store storages.

> 3 (1-false). What are the plans for SQL transaction support later on? Will
it likely be supported in the following 2.x versions?

Currently there is no (at least I don't know about it) the
transactional SQL in road map for 2.x.

> I've noticed Ignite
3.x initiative that regards tables as first-class citizens so that SQL
transactions and K-V transactions would be the same thing and natively be
supported?

Yes, Apache Ignite 3 goal is to provide transactions with strong
serializable consistency. KV API and SQL will use the same
transactional protocol and will not provide weak atomic guaranties.
Nevertheless, single KV read/write operations will have minimal
transactional overhead for most cases.

On Tue, Aug 2, 2022 at 3:26 PM Yijie Shen  wrote:
>
> Hi,
>
> I notice that "Caches with the TRANSACTIONAL_SNAPSHOT atomicity mode
> support SQL transactions"[1], but there are also notes on deprecation of `
> TRANSACTIONAL_SNAPSHOT` since Ignite 2.12 due to lots of limitations and
> are not considered production ready [2].
>
> I have a few questions about SQL transactions in Ignite:
>
> 1. Is MVCC the only atomicity mode that supports SQL transactions? What if
> I create a table with TRANSACTIONAL atomicity? Will it fall back to like
> optimistic/pessimistic transactions (OCC/PCC), or just no transaction
> supports at all? If OCC/PCC is applied, what are the limitations?
>
> 2 (1-yes). Is that possible to implement an external cache store that
> provides some sort of transaction support that could work for SQL OCC/PCC,
> any suggestions on supplemental materials that are helpful for the goal?
>
> 3 (1-false). What are the plans for SQL transaction support later on? Will
> it likely be supported in the following 2.x versions? I've noticed Ignite
> 3.x initiative that regards tables as first-class citizens so that SQL
> transactions and K-V transactions would be the same thing and natively be
> supported?
>
> Best,
> Yijie
>
> [1] https://ignite.apache.org/docs/latest/transactions/mvcc
> [2] https://issues.apache.org/jira/browse/IGNITE-15757

Re: URL parameters in CLI

[Andrey Gura]

The main question here is : what is the node URL? How will the user
know a port from a wide port range?

On Tue, Jul 26, 2022 at 3:01 PM Vadim Pakhnushev
 wrote:
>
> Hi Igniters!
> After implementing commands for CLI tool listed in the IEP-88
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool some
> early users raised a concern about confusion between --node-url and
> --cluster-url arguments.
> Initially --node-url was intended to be used in commands dealing with the
> particular node (i.e. node config show) and --cluster-url for commands
> dealing with the cluster as a whole (i.e. cluster config show).
> Although cluster URL is basically a URL of any node in the cluster.
> So the question for discussion is - would it make sense to unify parameters
> to both cluster- and node-related commands to be --node-url?
> Or it would be better to keep the distinction?

Re: [VOTE] Ignite Packaging IEP

[Andrey Gura]

Igniters,

What is the purpose of this vote? There is no problem during
discussion of the IEP. Silent consensus is enough here.

As I understand, aforementioned dependencies don't affect production
code, so this item doesn't require a voting process.

+1 from me. But it seems that we can go forward without any voting.


On Mon, Aug 29, 2022 at 10:01 AM Вячеслав Коптилин
 wrote:
>
> +1
>
> пт, 26 авг. 2022 г. в 18:07, Mikhail Pochatkin :
>
> > Hi! No, this doesn't affect Ignite 2.x. This IEP is applicable only for
> > Apache Ignite 3.
> >
> > On Fri, Aug 26, 2022 at 5:47 PM Ilya Kasnacheev  > >
> > wrote:
> >
> > > Hello!
> > >
> > > Please clarify whether this is Apache Ignite 3 only and that it does not
> > > affect Apache Ignite 2.x (or does it?)
> > >
> > > Thanks,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 22 авг. 2022 г. в 16:07, Mikhail Pochatkin :
> > >
> > > > Hi Igniters,
> > > >
> > > > I want to start a voting process about several points from IEP-93 [1].
> > > >
> > > > 1. Switch Apache Ignite build system to Gradle as default.
> > > > 2. Consequently, after changing the build system to Gradle we need to
> > add
> > > > new third party dependencies to several Gradle plugins. This is list of
> > > new
> > > > complete time deps:
> > > > openapiGenerator = "org.openapi.generator:5.4.0"
> > > > javacc = "com.intershop.gradle.javacc:4.0.1"
> > > > launch4j = "edu.sc.seis.launch4j:2.5.3"
> > > > shadow = "com.github.johnrengelman.shadow:7.1.2"
> > > > cmake = "io.github.tomtzook.gradle-cmake:1.0.0"
> > > > jreleaser =  "org.jreleaser:1.1.0"
> > > > 3. Agreement on all target package formats for Apache Ignite
> > > distributions.
> > > >
> > > > The vote is formal, see voting guidelines [3].
> > > >
> > > > +1 - to accept all points above
> > > > 0 - don’t care either way
> > > > -1 - DO NOT accept (explain why)
> > > >
> > > > This vote will be open for at least 4 days till Friday Aug 26, 2022,
> > > > 22:00 Moscow TZ.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-93%3A+Ignite+Packaging
> > > > [2] https://www.apache.org/foundation/voting.html
> > > >
> > >
> >

Apache Ignite 3.0.0 beta 1 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Hi, Igniters!

It's time for a new release of Apache Ignite 3 beta 1. The expected
feature list consists of:

- RPM and DEB packages: simplified installation and node management
with system services.
- Client's Partition Awareness: Clients are now aware of data
distribution over the cluster nodes which helps avoid additional
network transmissions and lowers operations latency.
- C++ client:  Basic C++ client, able to perform operations on data.
- Autogenerated values: now a function can be specified as a default
value generator during a table creation. Currently only
gen_random_uuid is supported.
- SQL Transactions.
- Transactional Protocol: improved locking model, multi-version based
lock-free read-only transactions.
- Storage: A number of improvements to memory-only and on-disk engines
based on Page Memory.
- Indexes: Basic functionality, hash and sorted indexes.
- Client logging: A LoggerFactory may be provided during client
creation to specify a custom logger for logs generated by the client.
- Metrics framework: Collection and export of cluster metrics.

I want to propose myself to be the release manager of the Apache
Ignite 3 beta 1.

Also I propose the following milestones for the release:

Scope Freeze: October 12, 2022
Code Freeze: October 20, 2022
Voting Date: October 31, 2022
Release Date: November 5, 2022

WDYT?

[WANTED A NEW RELEASE MANAGER] Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Igniters,

Due to personal reasons I need to take a pause so we need a new
release manager for the Apache Ignite 3 beta 1 release.

The best option is a PMC member. Committer is also a good option, but
it will need some help from PMC members.

Please feel free to submit your candidacy in this thread.

Re: Apache Ignite 3.0.0 beta 1 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Igniters,

I removed the "3.0.0-alpha6" version and created the "3.0.0-beta1"
version. All issues were rescheduled to the "3.0.0-beta1" version.

Despite the fact that formally was on 12th october there is still
possibility to add issues to the release scope. Deadline is 14th
october. Tomorrow I will announce the scope freeze officially. It
means that any issue could be added to the release only after
discussion with the community and a release manager.

On Mon, Oct 10, 2022 at 7:41 AM Aleksandr Pakhomov  wrote:
>
> +1
>
> > On 7 Oct 2022, at 23:05, Andrey Gura  wrote:
> >
> > Hi, Igniters!
> >
> > It's time for a new release of Apache Ignite 3 beta 1. The expected
> > feature list consists of:
> >
> > - RPM and DEB packages: simplified installation and node management
> > with system services.
> > - Client's Partition Awareness: Clients are now aware of data
> > distribution over the cluster nodes which helps avoid additional
> > network transmissions and lowers operations latency.
> > - C++ client:  Basic C++ client, able to perform operations on data.
> > - Autogenerated values: now a function can be specified as a default
> > value generator during a table creation. Currently only
> > gen_random_uuid is supported.
> > - SQL Transactions.
> > - Transactional Protocol: improved locking model, multi-version based
> > lock-free read-only transactions.
> > - Storage: A number of improvements to memory-only and on-disk engines
> > based on Page Memory.
> > - Indexes: Basic functionality, hash and sorted indexes.
> > - Client logging: A LoggerFactory may be provided during client
> > creation to specify a custom logger for logs generated by the client.
> > - Metrics framework: Collection and export of cluster metrics.
> >
> > I want to propose myself to be the release manager of the Apache
> > Ignite 3 beta 1.
> >
> > Also I propose the following milestones for the release:
> >
> > Scope Freeze: October 12, 2022
> > Code Freeze: October 20, 2022
> > Voting Date: October 31, 2022
> > Release Date: November 5, 2022
> >
> > WDYT?
>

Re: [WANTED A NEW RELEASE MANAGER] Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Slava, Stan,

formally Stan can't be a release manager because he isn't a committer. So
it would be better if Slava will be a release manager and Stan will assist
him.

It also means that all emails from Slava or Stan (announces, requests, etc)
related to the release will be valid.

чт, 13 окт. 2022 г., 15:36 Вячеслав Коптилин :

> Hi Stan,
>
> > I’ll need help from a PMC to do the steps requiring PMC permissions but I
> can do most of the work.
> I'm ready to assist you with this.
>
> Thanks,
> Slava.
>
>
> чт, 13 окт. 2022 г. в 15:08, :
>
> > Hi,
> >
> > I’ll be happy to do this.
> >
> > I’ll need help from a PMC to do the steps requiring PMC permissions but I
> > can do most of the work.
> >
> > Any PMC who is ready to support me in this?
> >
> > Thanks,
> > Stan
> >
> > > On 13 Oct 2022, at 15:54, Andrey Gura  wrote:
> > >
> > > Igniters,
> > >
> > > Due to personal reasons I need to take a pause so we need a new
> > > release manager for the Apache Ignite 3 beta 1 release.
> > >
> > > The best option is a PMC member. Committer is also a good option, but
> > > it will need some help from PMC members.
> > >
> > > Please feel free to submit your candidacy in this thread.
> >
>

Re: [WANTED A NEW RELEASE MANAGER] Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Stan, Slava,

I'll announce the scope freeze very soon. After that I'll announce
changes in the release management.

Thank you for your help!

On Thu, Oct 13, 2022 at 5:46 PM Kseniya Romanova  wrote:
>
> That's unusual, but doesn't seem to break any formal rules as I can see.
>
> чт, 13 окт. 2022 г. в 16:34, Stanislav Lukyanov :
>
> > Thanks Slava. For me, that's a dream team of two! :)
> > Happy to work in this way if there are no objections.
> >
> > Thanks,
> > Stan
> >
> > > On 13 Oct 2022, at 17:08, Andrey Gura  wrote:
> > >
> > > Slava, Stan,
> > >
> > > formally Stan can't be a release manager because he isn't a committer. So
> > > it would be better if Slava will be a release manager and Stan will
> > assist
> > > him.
> > >
> > > It also means that all emails from Slava or Stan (announces, requests,
> > etc)
> > > related to the release will be valid.
> > >
> > > чт, 13 окт. 2022 г., 15:36 Вячеслав Коптилин :
> > >
> > >> Hi Stan,
> > >>
> > >>> I’ll need help from a PMC to do the steps requiring PMC permissions
> > but I
> > >> can do most of the work.
> > >> I'm ready to assist you with this.
> > >>
> > >> Thanks,
> > >> Slava.
> > >>
> > >>
> > >> чт, 13 окт. 2022 г. в 15:08, :
> > >>
> > >>> Hi,
> > >>>
> > >>> I’ll be happy to do this.
> > >>>
> > >>> I’ll need help from a PMC to do the steps requiring PMC permissions
> > but I
> > >>> can do most of the work.
> > >>>
> > >>> Any PMC who is ready to support me in this?
> > >>>
> > >>> Thanks,
> > >>> Stan
> > >>>
> > >>>> On 13 Oct 2022, at 15:54, Andrey Gura  wrote:
> > >>>>
> > >>>> Igniters,
> > >>>>
> > >>>> Due to personal reasons I need to take a pause so we need a new
> > >>>> release manager for the Apache Ignite 3 beta 1 release.
> > >>>>
> > >>>> The best option is a PMC member. Committer is also a good option, but
> > >>>> it will need some help from PMC members.
> > >>>>
> > >>>> Please feel free to submit your candidacy in this thread.
> > >>>
> > >>
> >
> >

[ANNOUNCE] SCOPE FREEZE for Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Igniters,

The 'ignite-3.0.0-beta1' branch was created (the latest commit is
8160ef31ecf8d49f227562b6f0ab090c6b4438c1).

The scope for the release is frozen.

It means the following:

- Any issue could be added to the release (fixVersion == 3.0.0-beta1)
only after discussion with the community and a release manager in this
thread.
- Any commit to the release branch must be also applied to the 'main' branch.

[ANNOUNCE] Release managers for Apache Ignite 3.0.0 beta 1 RELEASE

[Andrey Gura]

Igniters,

Vyacheslav Koptilin () is a release manager
for the Apache Ignite 3.0.0 beta 1 release.

Stan Lukyanov () will assist Vyacheslav.

Slava, Stan, thanks a lot for your help!

Re: [VOTE] Release Extensions Ignite Spark 2.0.0 RC1, Ignite Spark 3.0.0 RC1

[Andrey Gura]

+1

On Fri, Dec 9, 2022 at 10:36 AM Nikita Amelchev  wrote:
>
> +1
>
> Checked compilation, ran some examples.
>
> пт, 9 дек. 2022 г. в 03:53, Alexandr Shapkin :
> >
> > Dear Community,
> >
> >
> > The release candidates are ready.
> >
> > These are new Ignite Spark integrations based on Spark 2.4 and 3.2 
> > dependencies respectively.
> >
> >
> > See the links below.
> >
> >
> > == Ignite Spark Extension Module 2.0.0 ==
> >
> > The release candidate is uploaded to:
> > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spark-ext-2.0.0-rc1/
> >
> > The following Maven staging can be used:
> > https://repository.apache.org/content/repositories/orgapacheignite-1557/org/apache/ignite/ignite-spark-ext/
> >
> > Tag:
> > https://github.com/apache/ignite-extensions/releases/tag/ignite-spark-ext-2.0.0-rc1
> >
> > RELEASE_NOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-2.0.0/modules/spark-ext/spark/RELEASE_NOTES.txt
> >
> > DEVNOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-2.0.0/DEVNOTES.md
> >
> > Release Checker Results:
> > https://github.com/apache/ignite-extensions/actions/runs/3652976657
> >
> >
> >
> > == Ignite Spark Extension Module 3.0.0 ==
> >
> > The release candidate is uploaded to:
> > https://dist.apache.org/repos/dist/dev/ignite/ignite-extensions/ignite-spark-ext-3.0.0-rc1/
> >
> > The following Maven staging can be used:
> > https://repository.apache.org/content/repositories/orgapacheignite-1557/org/apache/ignite/ignite-spark-ext/
> >
> > Tag:
> > https://github.com/apache/ignite-extensions/releases/tag/ignite-spark-ext-3.0.0-rc1
> >
> > RELEASE_NOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-3.0.0/modules/spark-ext/spark/RELEASE_NOTES.txt
> >
> > DEVNOTES:
> > https://github.com/apache/ignite-extensions/blob/release/ignite-spark-ext-3.0.0/DEVNOTES.md
> >
> > Release Checker Results:
> > https://github.com/apache/ignite-extensions/actions/runs/3634524603
> >
> >
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept (Ignite Spark 3.0.0 RC1, Ignite Spark 2.0.0 RC1)
> > 0 - don't care either way
> > -1 - DO NOT accept (explain why)
> >
> > See notes on how to verify release here
> > https://www.apache.org/info/verification.html
> > and
> > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
> >
> >
> > This vote will be open for at least 72 hours till Mon Dec, 12 2022, 02:00 
> > CET TZ.
> > Please, write down the thread if you need additional time to check the 
> > release.
> >
>
>
> --
> Best wishes,
> Amelchev Nikita

Re: [VOTE] Apache Ignite PMC Chair

[Andrey Gura]

+1 for Alexey Goncharuk (binding)

On Mon, Oct 28, 2019 at 9:07 PM Denis Magda  wrote:
>
> +1 for Nikolay Izhikov (binding)
>
> -
> Denis
>
>
> On Mon, Oct 28, 2019 at 11:06 AM Denis Magda  wrote:
>
> > Ignite community,
> >
> > Please cast a vote for one of the following candidates:
> >
> >- Alexey Goncharuk
> >- Dmitry Pavlov
> >- Nikolay Izhikov
> >- Pavel Tupitsyn
> >
> > Everybody is encouraged to take part in the vote, however, remember that
> > only the binding votes of PMC members result in the Chair election. The
> > candidates can vote for themselves. The vote will end on November 1st,
> > 11:00 AM Pacific Time. More details about the next steps are here:
> > https://www.apache.org/dev/pmc.html#newchair
> >
> > -
> > Denis
> >

Re: Lets add measure units for metrics

[Andrey Gura]

Units just can be added to a metric description. What is the purpose
of dedicated field for it?

We doesn't provide any API for work with measurement units and should
not do it. Metrics interpretation is responsibility of external
systems for metrics gathering. Enum with metrics measurement doesn't
bring any value for external systems and only useful thing is self
documentation.

On Wed, Nov 6, 2019 at 12:42 PM Nikolay Izhikov  wrote:
>
> Hello, Alexey.
>
> I'm +1.
> Let's do it.
>
> В Ср, 06/11/2019 в 16:37 +0700, Alexey Kuznetsov пишет:
> > Hi, All!
> >
> > I'm working on a tool for metrics monitoring.
> > And found that Metric interface [1] has a name and description, but has not
> > measurement units.
> >
> > When you are working on a tool that collects all metrics and shows it in
> > human readable format it is a very important piece of information.
> >
> > Because we have a lot of different metrics, some of them in bytes, some of
> > them in percents, some of them are simple counters and so on and so forth.
> >
> > How about to add Enum with all possible measurement units to  Metric
> > interface [1] ?
> >
> > Thoughts?
> >
> > [1]
> > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/metric/Metric.java
> >

Re: Lets add measure units for metrics

[Andrey Gura]

> Yes, we can have measurement units in description, but external tool will
be forced to parse textual representation of units.

We do not know anything about that tools. What kind of units they
support? What kind of actions can be done with measurement on the
external tool side? We can only guess about this. We can't even make
an exhaustive list of units.

So, from my point of view, it is task without formalized requirements.

And moreover, it is useless task because most tools just store and
display named values and provide user defined aggregations over that
values. It doesn't make sense to keeping measurement and transferring
measurement units between servers. Just redundant data and investing
to global warm up :) Units are about metric values interpretation. But
interpretation is out of scope for Ignite.

On Wed, Nov 6, 2019 at 2:48 PM Alexey Kuznetsov  wrote:
>
> Andrey,
>
> The main reason of separate field - is a simplicity of usage in tools.
> Yes, we can have measurement units in description, but external tool will
> be forced to parse textual representation of units.
> Description is a text that has no special format and how we can ask
> developers to not forget to specify units in description?
> But with enum we can make it mandatory field. Or write a test that will
> check all metrics for "not null" for this field.
>
> What do you think?
>
>
> On Wed, Nov 6, 2019 at 6:28 PM Andrey Gura  wrote:
>
> > Units just can be added to a metric description. What is the purpose
> > of dedicated field for it?
> >
> > We doesn't provide any API for work with measurement units and should
> > not do it. Metrics interpretation is responsibility of external
> > systems for metrics gathering. Enum with metrics measurement doesn't
> > bring any value for external systems and only useful thing is self
> > documentation.
> >
> > On Wed, Nov 6, 2019 at 12:42 PM Nikolay Izhikov 
> > wrote:
> > >
> > > Hello, Alexey.
> > >
> > > I'm +1.
> > > Let's do it.
> > >
> > > В Ср, 06/11/2019 в 16:37 +0700, Alexey Kuznetsov пишет:
> > > > Hi, All!
> > > >
> > > > I'm working on a tool for metrics monitoring.
> > > > And found that Metric interface [1] has a name and description, but
> > has not
> > > > measurement units.
> > > >
> > > > When you are working on a tool that collects all metrics and shows it
> > in
> > > > human readable format it is a very important piece of information.
> > > >
> > > > Because we have a lot of different metrics, some of them in bytes,
> > some of
> > > > them in percents, some of them are simple counters and so on and so
> > forth.
> > > >
> > > > How about to add Enum with all possible measurement units to  Metric
> > > > interface [1] ?
> > > >
> > > > Thoughts?
> > > >
> > > > [1]
> > > >
> > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/metric/Metric.java
> > > >
> >
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com

Re: Lets add measure units for metrics

[Andrey Gura]

Stephen,

All metrics will be local for node. Cluster wide metric is some
aggregation which could and must be done by special tools not Ignite.

We already have dozen cluster wide but meaningless metrics and a lot
of code for aggregation. It just isn't our job. All cluster wide
metrics will be removed in Apache Ignite 3.0 (or at least most of
them). Most metrics will be rethinked.

On Wed, Nov 6, 2019 at 3:44 PM Stephen Darlington
 wrote:
>
> Another thing that I would like to see in the metrics is the scope: is this 
> metric cluster-wide or only for this node?
>
> Like the units of measure, this could be an extra field. Or it could be 
> naming convention? (Currently many node-specific metrics have “local” in the 
> name, but this is not entirely consistent.) Either way, I agree that hiding 
> this information in a plain-text field is not a good idea; it should be 
> possible to determine programatically.
>
> Regards,
> Stephen
>
> > On 6 Nov 2019, at 12:48, Alexey Kuznetsov  wrote:
> >
> > Andrey,
> >
> > The main reason of separate field - is a simplicity of usage in tools.
> > Yes, we can have measurement units in description, but external tool will
> > be forced to parse textual representation of units.
> > Description is a text that has no special format and how we can ask
> > developers to not forget to specify units in description?
> > But with enum we can make it mandatory field. Or write a test that will
> > check all metrics for "not null" for this field.
> >
> > What do you think?
> >
> >
> > On Wed, Nov 6, 2019 at 6:28 PM Andrey Gura  wrote:
> >
> >> Units just can be added to a metric description. What is the purpose
> >> of dedicated field for it?
> >>
> >> We doesn't provide any API for work with measurement units and should
> >> not do it. Metrics interpretation is responsibility of external
> >> systems for metrics gathering. Enum with metrics measurement doesn't
> >> bring any value for external systems and only useful thing is self
> >> documentation.
> >>
> >> On Wed, Nov 6, 2019 at 12:42 PM Nikolay Izhikov 
> >> wrote:
> >>>
> >>> Hello, Alexey.
> >>>
> >>> I'm +1.
> >>> Let's do it.
> >>>
> >>> В Ср, 06/11/2019 в 16:37 +0700, Alexey Kuznetsov пишет:
> >>>> Hi, All!
> >>>>
> >>>> I'm working on a tool for metrics monitoring.
> >>>> And found that Metric interface [1] has a name and description, but
> >> has not
> >>>> measurement units.
> >>>>
> >>>> When you are working on a tool that collects all metrics and shows it
> >> in
> >>>> human readable format it is a very important piece of information.
> >>>>
> >>>> Because we have a lot of different metrics, some of them in bytes,
> >> some of
> >>>> them in percents, some of them are simple counters and so on and so
> >> forth.
> >>>>
> >>>> How about to add Enum with all possible measurement units to  Metric
> >>>> interface [1] ?
> >>>>
> >>>> Thoughts?
> >>>>
> >>>> [1]
> >>>>
> >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/metric/Metric.java
> >>>>
> >>
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
>
>

Re: Contribution

[Andrey Gura]

Hi, Mirza!

I've granted contributor role to your account.

Welcome on board.

On Mon, Nov 11, 2019 at 6:55 PM Мирза Алиев  wrote:
>
> Hello!
>
> I would like to start to contribute, could you please add me to
> contributors list? My login for https://issues.apache.org/ is maliev
>
> Best regards,
> Mirza Aliev

Re: ML Starter Tasks

[Andrey Gura]

It seems that JIRA have some troubles with link building.

Fixed link:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22Patch%20Available%22)%20AND%20component%20%3D%20ml%20AND%20labels%20%3D%20newbie%20ORDER%20BY%20priority%20DESC

On Tue, Nov 19, 2019 at 5:20 PM Alexey Zinoviev  wrote:
>
> Hi, Igniters and newbies from outer world.
> I've prepared today a few tickets for newbies under umbrella epic
> https://issues.apache.org/jira/browse/IGNITE-9281
>
> Also, all open newbie tickets in Ignite ML could be found by filter
> https://issues.apache.org/jira/browse/IGNITE-12383?jql=project%20%3D%20IGNITE%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22Patch%20Available%22)%20AND%20component%20%3D%20ml%20AND%20labels%20%3D%20newbie%20ORDER%20BY%20priority%20DESC
>
>
> If you are interested in ML development, write in this thread or assign one
> of the given tickets for himself.
>
> Also, options for .NET/Python/R/C++ API development are open for ML module,
> we could discuss them separately.

Re: Joining node validation failure event.

[Andrey Gura]

Hi, Mikhail!

Could you please describe the case for this new event?

On Wed, Nov 20, 2019 at 12:45 PM Mikhail Petrov  wrote:
>
> Hello, Igniters.
>
> There is a case which requires to handle joining node validation failure
> in Ignite components and obtain information of the node that tried to
> join and the reason for the failure. Now, as I see, there is no way to
> do it. I propose to implement a new event -- NodeValidationFailedEvent
> -- and record it in case the validation fails. I have created Tiket [1]
> and PR [2], which shows an example of implementation. Could anyone take
> a look at it, please?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12380
>
> [2] https://github.com/apache/ignite/pull/7057
>

Re: Joining node validation failure event.

[Andrey Gura]

Mikhail,

I don't understand how node validation on join affects security. But
it seems that you can use
PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode,
java.io.Serializable) method. Does it fit for your needs?

On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov  wrote:
>
> Hi, Ivan.
>
>
> Unfortunately, we came to no decision yet. As I mentioned above this
> event is disabled by default and no node will receive this event without
> an explicit subscription. In my case, that event is assumed to be used
> on node locally to share joining node info between security and
> discovery components. I have no idea how to solve this problem without
> publishing a new event too. But I'm concerned about the acceptance of
> that solution. Maybe it can be solved some other way? I will appreciate
> any suggestion or review PR [1] with event implementation.
>
>
> [1] https://github.com/apache/ignite/pull/7057
>
>
> Regards,
>
> Mikhail.
>
> On 02.12.2019 10:38, Ivan Pavlukhin wrote:
> > Mikhail, Andrey,
> >
> > Have you come to a common decision here? As for me, it sounds useful
> > to expose node join failure details somehow. The thing to decide on is
> > a mechanism to expose it. Unfortunately, immediately have no idea
> > better than using events.
> >
> >> What is purpose of the special cluster wide event? Node is not joined
> >> to the topology. Why topology nodes should know something about this
> >> node?
> > Was this question answered? I suppose that at least coordinator will
> > receive the event, will not it?
> >
> > чт, 28 нояб. 2019 г. в 10:10, Mikhail Petrov :
> >> Hi, Ivan.
> >>
> >> I'm sorry that the discussion was moved in private channel. The problem
> >> I'm trying to solve is described below in the thread. The security
> >> plugin in my case stores and analyzesinfo about a node that failed to join.
> >>
> >>
> >> Regards,
> >>
> >> Mikhail.
> >>
> >>
> >>
> >>  Forwarded Message 
> >> Subject:Re: Joining node validation failure event.
> >> Date:   Thu, 21 Nov 2019 21:43:33 +0300
> >> From:   Mikhail Petrov 
> >> To: Andrey Gura 
> >>
> >>
> >>
> >> Hi, Andrey.
> >>
> >> In my task security plugin running on the coordinator must locally
> >> handle the situation when some node is trying to join the topology with
> >> the invalid configuration. I can't handle the response on a node that
> >> failed to connect because it's untrusted.
> >>
> >> Actually I'm only concerned about one validation -- when all Ignite
> >> components validate new node.
> >>
> >> In my case plugin must be able to obtain general and security subject
> >> information from joining TcpDiscoveryNode attributes. But I have no idea
> >> how to share information between the security and discovery components
> >> without recording event and listening it locally.
> >>
> >> This event is assumed to be disable by default and in my case used only
> >> on local node so it's not look like "cluster wide event".
> >>
> >> Also I propose to record this event in dedicated utilityPool so it can't
> >> stuck discovery thread.
> >>
> >> I will appreciate any thoughts on this problem.
> >>
> >> Regards,
> >> Mikhail.
> >>
> >> On 21.11.2019 19:40, Andrey Gura wrote:
> >>> Mikhail,
> >>>
> >>> It is still not enough details to me. What is expected behavior if the
> >>> plugin?
> >>>
> >>> There are a different validations during node join. Many of them
> >>> placed in RingMessageWorker#processJoinRequestMessage method. If
> >>> validation will fail then corresponding message will be sent to the
> >>> joining node (including TcpDiscoveryAuthFailedMessage) and node will
> >>> not joined to topology.
> >>>
> >>> What is purpose of the special cluster wide event? Node is not joined
> >>> to the topology. Why topology nodes should know something about this
> >>> node?
> >>>
> >>> On Thu, Nov 21, 2019 at 9:54 AM Mikhail Petrov 
> >>> wrote:
> >>>> Hi, Andrey.
> >>>>
> >>>> I take part in the development of a custom security plugin for Apache
> >>>> Ignite. There is an information security requirement for which n

Re: Cache operations performance metrics

[Andrey Gura]

>From my point of view, Ignite should provide meaningful metrics for
internal components that could be useful for monitoring and analysis.
All suggested options are meaningless in a sense. Below I'll try
explain why.

>* `get`, `put`, `remove` time histograms. Measured for API calls on the caller 
>node side.
>Implemented in [1], commit [2].

All cache operations in Ignite are distributed. So each value measured
for some cache operation will vary depending on where actually
operation is performed. Values will the same only for cases when node
is remote relative to data (e.g. client node).

For regular data node (server node) timing will depend on answers for question:

- is node primary for particular key or not? (for all operations)
- how many backups configured for the cache? (for put and remove)
- what write synchronization mode is configured for particular cache?
(for put and remove)
- is readFromBackup enabled for the cache? (for get)

Both Ignite users and Ignite developers can't make any decision based
on this metrics.

> * `commit`, `rollback` time histograms. Measured for API calls on the caller 
> node side [3].

What is transaction commit or rollback time? How it calculates in
Ignite now? What actions included into transaction? What actions not
related with cache executed during transactions?

There is no any sense in time of transaction commit or rollback
because there are no any way to understand what transaction was
performed in particular period of time. Usually a lot of transactions
and we can't to distinguish from each other.

Moreover, transaction usually treats as business operation. So only
way to measure performance properly is measure business operation
time. That is user should create own metrics set for some business
API.

Further. What about cross cache transactions? At the moment tx
commit/rollback time will be added to corresponding metrics per each
cache evolved to the transaction. The *same time* for *each cache*.
Absolutely meaningless.

Again, both Ignite users and Ignite developers can't make any decision
based on this metrics. But users can create own metrics set.

>* histograms that measure the time of processing `get`, `put`, `remove`, 
>`commit`, `rollback` messages on affinity nodes(primary and backups).
>Ticket doesn't exist for it.

It will be implemented for most types of messages.

Metrics, application monitoring, performance analysis and measurement
are a a little harder than it sounds. Therefore, we must approach this
issue more carefully.
Blindly adding new types of metrics will not only not improve the
situation, but will also worsen the overall performance of the system
because metric calculation always on the hot path.

So, from my point of view, commits for get/put/remove and
commit/rollback should be reverted.

On Mon, Dec 16, 2019 at 5:39 PM Nikita Amelchev  wrote:
>
> I think these metrics are useful.
>
> I have prepared PR [1] for commit and rollback histograms. [2]
> Nikolay, could you take a look, please?
>
> If you do not mind, I will try to add affinity-nodes cache metrics:
> >> * histograms that measure the time of processing `get`, `put`, `remove`, 
> >> `commit`, `rollback` messages on affinity nodes(primary and backups). 
> >> Ticket doesn't exist for it.
>
> I have filed a ticket for it. [3]
>
> [1] https://github.com/apache/ignite/pull/7141
> [2] https://issues.apache.org/jira/browse/IGNITE-12450
> [3] https://issues.apache.org/jira/browse/IGNITE-12453
>
> пн, 16 дек. 2019 г. в 11:07, Alexei Scherbakov :
> >
> > I think they are very useful.
> >
> > пн, 16 дек. 2019 г. в 10:51, Николай Ижиков :
> >
> > > Hello, Alexei.
> > >
> > > Thanks for the link on the ticket, lableled it with the IEP-35 label.
> > > What do you think about proposed metrics set?
> > >
> > > > 16 дек. 2019 г., в 10:29, Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com> написал(а):
> > > >
> > > > Nikolay,
> > > >
> > > > What about batch operations?
> > > >
> > > > For messages processing the ticket does exist and even has an
> > > > implementation from before new metrics API times [1]
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-10418
> > > >
> > > > пн, 16 дек. 2019 г. в 10:12, Николай Ижиков :
> > > >
> > > >> Hello, Igniters.
> > > >>
> > > >> I want to provide the user answers to the following question: "How 
> > > >> cache
> > > >> API operations perform?"
> > > >> It seems, we need to implements metrics for basic cache API operations
> > > >> like get, put, remove for it.
> > > >>
> > > >> I think we should provide the following metrics:
> > > >>
> > > >> * `get`, `put`, `remove` time histograms. Measured for API calls on the
> > > >> caller node side.
> > > >>Implemented in [1], commit [2].
> > > >>
> > > >> * `commit`, `rollback` time histograms. Measured for API calls on the
> > > >> caller node side [3].
> > > >>
> > > >> * histograms that measure the time of processing `get`, `put`, 
> > > >> `remove`,
> > > >> `commit`, `roll

Checkstyle "NewlineAtEndOfFile" rule issue

[Andrey Gura]

Igniters,

recently I run build with checkstyle profile on Windows machine and
got 8 issues related to the "NewlineAtEndOfFile" rule while there are
no problems on my Linux machine.

I investigated the problem and suggest explicitly configure this rule
by LF separator. See [1] for more details.

I want merge this change if there are no any objections from community members.

[1] https://issues.apache.org/jira/browse/IGNITE-12472

Re: Checkstyle "NewlineAtEndOfFile" rule issue

[Andrey Gura]

Agree with Pavel. But it could broke something in our pipeline.

Maxim, if issue was fixed so it dependency update is better way. But
it seems that this issue in Open status.

On Thu, Dec 19, 2019 at 7:22 PM Pavel Tupitsyn  wrote:
>
> Maxim,
>
> The guidelines are not set in stone.
> If we decide that some guideline does not bring any value and only wastes
> our time (like this one),
> we can (and should) remove it.
>
> On Thu, Dec 19, 2019 at 7:13 PM Maxim Muzafarov  wrote:
>
> > Pavel,
> >
> > It's configured according to accepted Coding Guidelines [1].
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines#CodingGuidelines-Whitespacesandemptylines
> >
> > On Thu, 19 Dec 2019 at 18:59, Pavel Tupitsyn  wrote:
> > >
> > > Igniters,
> > >
> > > Does this rule bring any value whatsoever for us?
> > > Let's just disable it.
> > >
> > > On Thu, Dec 19, 2019 at 6:25 PM Maxim Muzafarov 
> > wrote:
> > >
> > > > Hello, Andrey
> > > >
> > > > Is it better to upgrade the checkstyle plugin version?
> > > > It seems the issue has been fixed since 8.21 version (currently we have
> > > > 8.19)
> > > >
> > > > [1] https://checkstyle.org/releasenotes.html#Release_8.21
> > > >
> > > > On Thu, 19 Dec 2019 at 18:09, Andrey Gura  wrote:
> > > > >
> > > > > Igniters,
> > > > >
> > > > > recently I run build with checkstyle profile on Windows machine and
> > > > > got 8 issues related to the "NewlineAtEndOfFile" rule while there are
> > > > > no problems on my Linux machine.
> > > > >
> > > > > I investigated the problem and suggest explicitly configure this rule
> > > > > by LF separator. See [1] for more details.
> > > > >
> > > > > I want merge this change if there are no any objections from
> > community
> > > > members.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12472
> > > >
> >

Re: Cache operations performance metrics

[Andrey Gura]

> The goal of the proposed metrics is to measure whole cache operations 
> behavior.
> It provides some kind of statistics(histograms) for it.

Nikolay, reformulating doesn't make metrics more meaningful. Seriously :)

> Yes, metrics will evaluate API call performance

And what? Where the sense in this value? I explained why this metrics
are relatively useless.

> These are metrics of client-side operation performance.

Again. It's just a number without any sense.

> I think a specific user has knowledge - what are his transactions.

May be. But user can't distinguish one transaction from another, so
his knowledge doesn't make sense definitely.

> From these metrics it can answer on the question «If my transaction includes 
> cacheXXX, how long it usually takes?»

Actually not. The same caches can be involved  in a dozen of
transactions and there are no ways to understand what transactions are
slow or fast. It is useless.

> I disagree here.
> If you have a better approach to measure cache operations performance - 
> please, share your vision.

I already wrote about better approach. Two main points:

1. Measure some important internals (WAL operations, checkpoint time,
etc) because it can talk about real problems.
2. Measure business operations in user context, not cache API operations.

So  what we have? We have useless metrics that are doubled by useless
histograms.

We should reconsider approach to metrics and performance measuring. It
is hard and long task. There are no need to commit tons of useless
metrics that just decrease performance.

Sorry for some sarcasm but I really believe in my opinion. Metrics
problem exists very very long time and existing metrics discussed many
times. No one can explain this metrics to users because it requires
too many additional knowledge about internals. And metric  value
itself depends on many aspects of internals. It leads to impossibility
of interpretation. And it's good time to remove it (in AI 3.0 due to a
backward compatibility).

On Thu, Dec 19, 2019 at 9:09 PM Николай Ижиков  wrote:
>
> Hello, Andrey.
>
> The goal of the proposed metrics is to measure whole cache operations 
> behavior.
> It provides some kind of statistics(histograms) for it.
> For more fine-grained analysis one will be use tracing or other «go deeper» 
> tools.
>
> > > Measured for API calls on the caller node side
> > Values will the same only for cases when node is remote relative to data
>
> Yes, metrics will evaluate API call performance.
> I think this is the most valuable information from a user's point of view.
>
> Regular user wants to know how fast his cache operation performs.
> And these metrics provide the answer.
>
> > For regular data node (server node) timing will depend on answers for 
> > question:
>
> I think these answers are always available.
> I barely can imagine a scenario when one monitor «black box» cluster and 
> don’t know it.
> Even so, all answers are provided through system view we brought to the 
> Ignite :)
>
> > What is transaction commit or rollback time?
>
> These are metrics of client-side operation performance.
>
> I think a specific user has knowledge - what are his transactions.
> From these metrics it can answer on the question «If my transaction includes 
> cacheXXX, how long it usually takes?»
> I think it’s very valuable knowledge.
>
> > It will be implemented for most types of messages.
>
> Good, let’s do it?
>
> > So, from my point of view, commits for get/put/remove and commit/rollback 
> > should be reverted.
>
> I disagree here.
> If you have a better approach to measure cache operations performance - 
> please, share your vision.
>
> > 19 дек. 2019 г., в 16:03, Andrey Gura  написал(а):
> >
> > From my point of view, Ignite should provide meaningful metrics for
> > internal components that could be useful for monitoring and analysis.
> > All suggested options are meaningless in a sense. Below I'll try
> > explain why.
> >
> >> * `get`, `put`, `remove` time histograms. Measured for API calls on the 
> >> caller node side.
> >>   Implemented in [1], commit [2].
> >
> > All cache operations in Ignite are distributed. So each value measured
> > for some cache operation will vary depending on where actually
> > operation is performed. Values will the same only for cases when node
> > is remote relative to data (e.g. client node).
> >
> > For regular data node (server node) timing will depend on answers for 
> > question:
> >
> > - is node primary for particular key or not? (for all operations)
> > - how many backups configured for the cache? (for put and remove)
> > - what w

Re: Cache operations performance metrics

[Andrey Gura]

> If a cache has some percent of the relatively slow transaction this is a 
> trigger to make a deeper investigation.

It also will be visible on other metrics. So cache operations metrics
still useless because it transitive values.

>> 1. Measure some important internals (WAL operations, checkpoint time, etc) 
>> because it can talk about real problems.

> We already implement it.

I don't talk that it isn't implemented. It is just example of things
that should be measured. All other metrics depends on internals.

>> 2. Measure business operations in user context, not cache API operations.

>Why do you think these approaches should exclude one another?

Because one of them is useless.

On Fri, Dec 20, 2019 at 1:43 PM Николай Ижиков  wrote:
>
> Hello, Andrey.
>
> > Where the sense in this value? I explained why this metrics are relatively 
> > useless.
>
> I don’t agree with you.
> I believe they are not useless for a user.
> And I try to explain why I think so.
>
> > But user can't distinguish one transaction from another, so his knowledge 
> > doesn't make sense definitely.
>
> Users shouldn’t distinguish.
> If a cache has some percent of the relatively slow transaction this is a 
> trigger to make a deeper investigation.
>
> > 1. Measure some important internals (WAL operations, checkpoint time, etc) 
> > because it can talk about real problems.
>
> We already implement it.
> What metrics are missing for internal processes?
>
> > 2. Measure business operations in user context, not cache API operations.
>
> Why do you think these approaches should exclude one another?
> Users definitely should measure whole business transaction performance.
>
> I think we should provide a way to measure part of the business transaction 
> that relates to the Ignite.
>
>
> > 20 дек. 2019 г., в 13:02, Andrey Gura  написал(а):
> >
> >> The goal of the proposed metrics is to measure whole cache operations 
> >> behavior.
> >> It provides some kind of statistics(histograms) for it.
> >
> > Nikolay, reformulating doesn't make metrics more meaningful. Seriously :)
> >
> >> Yes, metrics will evaluate API call performance
> >
> > And what? Where the sense in this value? I explained why this metrics
> > are relatively useless.
> >
> >> These are metrics of client-side operation performance.
> >
> > Again. It's just a number without any sense.
> >
> >> I think a specific user has knowledge - what are his transactions.
> >
> > May be. But user can't distinguish one transaction from another, so
> > his knowledge doesn't make sense definitely.
> >
> >> From these metrics it can answer on the question «If my transaction 
> >> includes cacheXXX, how long it usually takes?»
> >
> > Actually not. The same caches can be involved  in a dozen of
> > transactions and there are no ways to understand what transactions are
> > slow or fast. It is useless.
> >
> >> I disagree here.
> >> If you have a better approach to measure cache operations performance - 
> >> please, share your vision.
> >
> > I already wrote about better approach. Two main points:
> >
> > 1. Measure some important internals (WAL operations, checkpoint time,
> > etc) because it can talk about real problems.
> > 2. Measure business operations in user context, not cache API operations.
> >
> > So  what we have? We have useless metrics that are doubled by useless
> > histograms.
> >
> > We should reconsider approach to metrics and performance measuring. It
> > is hard and long task. There are no need to commit tons of useless
> > metrics that just decrease performance.
> >
> > Sorry for some sarcasm but I really believe in my opinion. Metrics
> > problem exists very very long time and existing metrics discussed many
> > times. No one can explain this metrics to users because it requires
> > too many additional knowledge about internals. And metric  value
> > itself depends on many aspects of internals. It leads to impossibility
> > of interpretation. And it's good time to remove it (in AI 3.0 due to a
> > backward compatibility).
> >
> > On Thu, Dec 19, 2019 at 9:09 PM Николай Ижиков  
> > wrote:
> >>
> >> Hello, Andrey.
> >>
> >> The goal of the proposed metrics is to measure whole cache operations 
> >> behavior.
> >> It provides some kind of statistics(histograms) for it.
> >> For more fine-grained analysis one will be use tracing or other «go 
> >> dee

Re: Cache operations performance metrics

[Andrey Gura]

> For example, the user saw «checkpoint time» metric becomes x2 bigger.

I just quote your words: " this is a trigger to make a deeper
investigation". And if we have two metrics that are triggered for the
same then one of them is useless.

> How it relates to business operations?

Why it should be related with business operation? It is concrete
metrics for concrete process which can slowdown all operations in the
system. Obviously if you want know how fast or slow your business
operation you must measure latency of your business operation. What
could be easier?

> Is it become slower or faster?

Very correct question! User saw "cache put time" metric becomes x2
bigger. Does it become slower or faster? Or we just put into the cache
values that 4x bigger in size? Or all time before we put values
locally and now we put values on remote nodes. Or our operations
execute in transaction and then time will depend on transaction type,
actions in transaction and other transaction and actually will nothing
talk about real cache operation. We have more questions then answers.

> On the other hand - if `PuTime` increased - then we know for sure, all 
> operation executing `put` becomes slower.

Of course not :) See above.

On Fri, Dec 20, 2019 at 3:20 PM Николай Ижиков  wrote:
>
> > It also will be visible on other metrics
>
> How will it be visible?
>
> For example, the user saw «checkpoint time» metric becomes x2 bigger.
> How it relates to business operations? Is it become slower or faster?
> What does it mean for an application performance?
>
> On the other hand - if `PuTime` increased - then we know for sure, all 
> operation executing `put` becomes slower.
>
> *Why* it’s become slower - is the essence of «go deeper» investigation.
>
> > 20 дек. 2019 г., в 15:07, Andrey Gura  написал(а):
> >
> >> If a cache has some percent of the relatively slow transaction this is a 
> >> trigger to make a deeper investigation.
> >
> > It also will be visible on other metrics. So cache operations metrics
> > still useless because it transitive values.
> >
> >>> 1. Measure some important internals (WAL operations, checkpoint time, 
> >>> etc) because it can talk about real problems.
> >
> >> We already implement it.
> >
> > I don't talk that it isn't implemented. It is just example of things
> > that should be measured. All other metrics depends on internals.
> >
> >>> 2. Measure business operations in user context, not cache API operations.
> >
> >> Why do you think these approaches should exclude one another?
> >
> > Because one of them is useless.
> >
> > On Fri, Dec 20, 2019 at 1:43 PM Николай Ижиков  wrote:
> >>
> >> Hello, Andrey.
> >>
> >>> Where the sense in this value? I explained why this metrics are 
> >>> relatively useless.
> >>
> >> I don’t agree with you.
> >> I believe they are not useless for a user.
> >> And I try to explain why I think so.
> >>
> >>> But user can't distinguish one transaction from another, so his knowledge 
> >>> doesn't make sense definitely.
> >>
> >> Users shouldn’t distinguish.
> >> If a cache has some percent of the relatively slow transaction this is a 
> >> trigger to make a deeper investigation.
> >>
> >>> 1. Measure some important internals (WAL operations, checkpoint time, 
> >>> etc) because it can talk about real problems.
> >>
> >> We already implement it.
> >> What metrics are missing for internal processes?
> >>
> >>> 2. Measure business operations in user context, not cache API operations.
> >>
> >> Why do you think these approaches should exclude one another?
> >> Users definitely should measure whole business transaction performance.
> >>
> >> I think we should provide a way to measure part of the business 
> >> transaction that relates to the Ignite.
> >>
> >>
> >>> 20 дек. 2019 г., в 13:02, Andrey Gura  написал(а):
> >>>
> >>>> The goal of the proposed metrics is to measure whole cache operations 
> >>>> behavior.
> >>>> It provides some kind of statistics(histograms) for it.
> >>>
> >>> Nikolay, reformulating doesn't make metrics more meaningful. Seriously :)
> >>>
> >>>> Yes, metrics will evaluate API call performance
> >>>
> >>> And what? Where the sense in this value? I explained why this metrics
> >>> are relatively useless.
> >>&

Re: Cache operations performance metrics

[Andrey Gura]

>> And if we have two metrics that are triggered for the same then one of them 
>> is useless.
> I don't understand what is the two metrics you are talking about.

Please, don't loose context. In your example it was checkpoint time
and some cache operation time.

> A business transaction includes work with several data sources, sending 
> network requests, executing some remote services.
> If it becomes slower then we should know - what basic API operations become 
> slower.

No, we should now what basic operations become slower. It is problem
with network (net io), with disk (disk io), JVM (VM internal metrics),
etc. All this operations are bricks of API operations.

> So we have to measure  `PutTime` from Ignite, `InsertTime` from RDBMS and 
> other parts of a transaction.

We can't do it properly due to a transactions implementation specific.
I already wrote about it. It doesn't mean that we must not fix it. But
it also means that we should reconsider approach to metrics in Ignite.
Bigger doesn't mean better.

> Ignite cache operations obviously becomes 2 times slower.
> *Why* they become slower is the question of an ongoing investigation.

But business operations metrics will show the same. And many other
internals related metrics will show the same. It is transitive,
redundant and relatively useless metric if it doesn't bring something
new in information. 500 caches with similar configurations (the same
nodes, the same data region, the same affinity, etc) and 500 metrics
like put time will show the same trend.And the same trend will show a
couple of system internal metrics. A couple vs hundreds. Doesn't make
sense and useless.

> I tried to look at other open-source products.
> Here is an example of metrics provided by Apache Kafka [1] [2]

If somebody do something it doesn't mean that they do it properly.

On Fri, Dec 20, 2019 at 4:28 PM Николай Ижиков  wrote:
>
> > And if we have two metrics that are triggered for the same then one of them 
> > is useless.
>
> I don't understand what is the two metrics you are talking about.
> I wrote about a single metric for a single cache operation.
>
> > Obviously if you want know how fast or slow your business operation you 
> > must measure latency of your business operation. What could be easier?
>
> A business transaction includes work with several data sources, sending 
> network requests, executing some remote services.
> If it becomes slower then we should know - what basic API operations become 
> slower.
> So we have to measure  `PutTime` from Ignite, `InsertTime` from RDBMS and 
> other parts of a transaction.
>
> Ignite will provide this kind of value out of the box.
> I think it’s useful values.
>
> > User saw "cache put time" metric becomes x2 bigger. Does it become slower 
> > or faster? Or we just put into the cache values that 4x bigger in size?
>
> Ignite cache operations obviously becomes 2 times slower.
> *Why* they become slower is the question of an ongoing investigation.
>
> I tried to look at other open-source products.
> Here is an example of metrics provided by Apache Kafka [1] [2]
>
> `request-latency-avg` - The average request latency in ms.
> `records-lag-max` - The maximum lag in terms of number of records for any 
> partition in this window. An increasing value over time is your best 
> indication that the consumer group is not keeping up with the producers.
> `fetch-latency-avg` - The average time taken for a fetch request.
>
> It seems, they implement a similar approach to what I proposed.
>
>
> [1] https://docs.confluent.io/current/kafka/monitoring.html#producer-metrics
> [2] 
> https://docs.confluent.io/current/kafka/monitoring.html#new-consumer-metrics
>
> > 20 дек. 2019 г., в 15:53, Andrey Gura  написал(а):
> >
> >> For example, the user saw «checkpoint time» metric becomes x2 bigger.
> >
> > I just quote your words: " this is a trigger to make a deeper
> > investigation". And if we have two metrics that are triggered for the
> > same then one of them is useless.
> >
> >> How it relates to business operations?
> >
> > Why it should be related with business operation? It is concrete
> > metrics for concrete process which can slowdown all operations in the
> > system. Obviously if you want know how fast or slow your business
> > operation you must measure latency of your business operation. What
> > could be easier?
> >
> >> Is it become slower or faster?
> >
> > Very correct question! User saw "cache put time" metric becomes x2
> > bigger. Does it become slower or faster? Or we just put into the cache
> > values that 4x bigger in s

Re: Re[2]: Cache operations performance metrics

[Andrey Gura]

> but between to have something and have nothing i choose — something

We already have "something". put, get, etc. metrics. As I told early
it relatively useless. But the same metrics with histograms doesn't
add any value.

> i found 1 grid machine with very different io usage than others, «dig deeper» 
> highlight cache with very different from other nodes cache put operations and 
> final «dig deeper» help to found code bug

I believe the same could be noticed using PK index stats.

> if new one would be more useful — why not ?

If some particular value is relatively useless then the same histogram
will be still relatively useless :) It's my point. Stop adding a dozen
of metrics, start thinking about benefits and meaning. Discuss it with
community.


On Fri, Dec 20, 2019 at 4:59 PM Zhenya Stanilovsky
 wrote:
>
>
> >> Is it become slower or faster?
> >
> >Very correct question! User saw "cache put time" metric becomes x2
> >bigger. Does it become slower or faster? Or we just put into the cache
> >values that 4x bigger in size? Or all time before we put values
> >locally and now we put values on remote nodes. Or our operations
> >execute in transaction and then time will depend on transaction type,
> >actions in transaction and other transaction and actually will nothing
> >talk about real cache operation. We have more questions then answers.
>
> Andrey, i hope i understand your point of view here, but between to have 
> something and have nothing i choose — something, it sometimes really helpful. 
> From real life case: i found 1 grid machine with very different io usage than 
> others, «dig deeper» highlight cache with very different from other nodes 
> cache put operations and final «dig deeper» help to found code bug, but to be 
> clear — old mechanism works ok for me here, if new one would be more useful — 
> why not ?
>
> >> On the other hand - if `PuTime` increased - then we know for sure, all 
> >> operation executing `put` becomes slower.
> >
> >Of course not :) See above.
> >
> >On Fri, Dec 20, 2019 at 3:20 PM Николай Ижиков < nizhi...@apache.org > wrote:
> >>
> >> > It also will be visible on other metrics
> >>
> >> How will it be visible?
> >>
> >> For example, the user saw «checkpoint time» metric becomes x2 bigger.
> >> How it relates to business operations? Is it become slower or faster?
> >> What does it mean for an application performance?
> >>
> >> On the other hand - if `PuTime` increased - then we know for sure, all 
> >> operation executing `put` becomes slower.
> >>
> >> *Why* it’s become slower - is the essence of «go deeper» investigation.
> >>
> >> > 20 дек. 2019 г., в 15:07, Andrey Gura < ag...@apache.org > написал(а):
> >> >
> >> >> If a cache has some percent of the relatively slow transaction this is 
> >> >> a trigger to make a deeper investigation.
> >> >
> >> > It also will be visible on other metrics. So cache operations metrics
> >> > still useless because it transitive values.
> >> >
> >> >>> 1. Measure some important internals (WAL operations, checkpoint time, 
> >> >>> etc) because it can talk about real problems.
> >> >
> >> >> We already implement it.
> >> >
> >> > I don't talk that it isn't implemented. It is just example of things
> >> > that should be measured. All other metrics depends on internals.
> >> >
> >> >>> 2. Measure business operations in user context, not cache API 
> >> >>> operations.
> >> >
> >> >> Why do you think these approaches should exclude one another?
> >> >
> >> > Because one of them is useless.
> >> >
> >> > On Fri, Dec 20, 2019 at 1:43 PM Николай Ижиков < nizhi...@apache.org > 
> >> > wrote:
> >> >>
> >> >> Hello, Andrey.
> >> >>
> >> >>> Where the sense in this value? I explained why this metrics are 
> >> >>> relatively useless.
> >> >>
> >> >> I don’t agree with you.
> >> >> I believe they are not useless for a user.
> >> >> And I try to explain why I think so.
> >> >>
> >> >>> But user can't distinguish one transaction from another, so his 
> >> >>> knowledge doesn't make sense definitely.
> >> >>
> >> >> Users shouldn’t distinguish.
> >&

Re: Internal classes are exposed in public API

[Andrey Gura]

Hi,

The first, I agree with Alexey about deprecation of APIs that are
still supported and don't offer reasonable substitution.

The second, from my point of view, we can't recommend
MetricExporterSpi's because it are still not-production ready. There
are some issues with it and usage of ReadOnlyMetricRegistry interface
just one of them.

The third, moving of MetricRegistry to the public API doesn't solve
the problem because this interface exposes internal Metric interface
implementations. So your PR is incomplete.
Moreover, API of MetricRegistry is inconsistent. E.g. register(name,
supplier, desc) method returns registered metric for some types and
doesn't for other. register(metric) method is inconsistent in sense of
metric naming. ReadOnlyMetricRegistry interface is redundant.
MetricExporterSpi should be revised because it absolutely not
intuitive because it requires ReadOnlyMetricRegistry and it's purpose
is undefined.

One more point. IEP-35 is still not fully implemented. Some things are
not taken into account. Exporters expose metrics if they are disabled.
JMX beans exposes values that don't confirm to best practices [1].
Metrics of type lists are not metric at all. Ubiquitous merics lookup
from hash map instead of usage reference for getting metrics values
(it is just performance issue). Also IGNITE-11927 is not implemented
which also changes interfaces significantly.

Let's just admit that the implementation is immature and must be moved
to the internal packages.

And because we already merged partially implemented IEP to the master
branch we *must move all currently public APIs to the internal API*
while it will not be ready for publication.

And the last but not least. What is happening indicates a immature
development process which must be revised. I don't want discuss it in
this thread but we must not allow merge of change to the master branch
before it will completed, that is we must use feature branches for
full IEP not only for particular tickets. And also we should
reformulate IEP process in order to avoid things like this.

[1] 
https://www.oracle.com/technetwork/java/javase/tech/best-practices-jsp-136021.html

On Fri, Jan 17, 2020 at 12:49 PM Николай Ижиков  wrote:
>
> Alex.
>
> OK, I may leverage your experience and create pure Java API.
> Ticket [1] created.
>
> But, personally, I don’t agree with you.
> Ignite has dozens of the API that theoretically have a usage scenario, but in 
> real-world have 0 custom implementation and usages.
> Moreover, many APIs that were created with the intentions you mentioned is 
> abandoned now and confuses users.
>
> You can just see count of the tests we just mute on the TC.
>
> Can you, please, take a look at the fix regarding puck API issue you 
> mentioned in your first letter [2], [3]
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12553
> [2] https://github.com/apache/ignite/pull/7269
> [3] https://issues.apache.org/jira/browse/IGNITE-12552
>
>
> > 17 янв. 2020 г., в 12:12, Alexey Goncharuk  
> > написал(а):
> >
> > Nikolay,
> >
> > Why do you think this is a wrong usage pattern? From the top of my head,
> > here is a few cases of direct metric API usage that I know are currently
> > being used in production:
> > * A custom task execution scheduling service with load balancing based on
> > utilization metrics readings from Java code
> > * Cleanup task trigger based on metrics readings
> > * A custom health-check endpoint for an application with an embedded
> > Ignite node for Kubernetes/Spring Application/etc
>