[MTCGA]: new failures in builds [6444493] needs to be handled

[ignitetcbot]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *Test with high flaky rate in master-nightly 
IgniteClusterActivateDeactivateTestWithPersistence.testDeactivateDuringEvictionAndRebalance
 
https://ci2.ignite.apache.org/test/-3681306600883946545?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 08:08:22 23-05-2022 

Re: IEP-90 Ignite 3 Client Lifecycle

[Andrey Gura]

Thanks for the answers, Igor.

On Thu, May 19, 2022 at 10:55 PM Igor Sapego  wrote:
>
> Andrey,
>
> 1. If a server generates a UUID that already exists it can check and just
> re-generate it straight away
> as a check is just a simple map lookup.
>
> 2. Well, yes. This proposal does not consider monitoring, but with current
> approach it will be definitely
> easier to implement it properly.
>
> Pavel,
>
> Wow, that's really cool that you've already started working on it.
>
> Regarding your proposal I like it - it will really make a procedure easier
> and faster for a client. I'll change
> the IEP accordingly.
>
> Best Regards,
> Igor
>
>
> On Thu, May 19, 2022 at 11:40 PM Pavel Tupitsyn 
> wrote:
>
> > Igor,
> >
> > I've started working on the initial sessions implementation [1],
> > and I'd like to clarify the procedure of logical connection restore.
> >
> > > client in its turn tries to establish a new *node connection*
> > > and restore *logical connection *using ConnectionRestoreReq
> >
> > This implies either an additional request, or something that replaces
> > normal handshake.
> > However, we need to handle two cases here:
> > - Logical connection is restored
> > - Logical connection is not restored (timed out, server restarted, etc), in
> > which case we still establish the connection and use it.
> >
> > To account for the second case, we should always start with a regular
> > handshake.
> > I propose to add a nullable UUID field to the handshake request to cover
> > both cases:
> > - connectionId is null or not found on server: proceed with normal
> > handshake, return newly generated connectionId.
> > - connectionId is not null and found on server: restore logical connection,
> > return the same connectionId
> >
> > Client checks if returned connectionId equals to the original and
> > understands whether logical connection was restored or not.
> >
> > Thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-16928
> >
> > On Thu, May 19, 2022 at 10:27 PM Pavel Tupitsyn 
> > wrote:
> >
> > > Andrey,
> > >
> > > > different connections on different client instances theoretically
> > > > could generate an already existing connection ID
> > >
> > > Do you mean an UUID collision?
> > >
> > > On Thu, May 19, 2022 at 1:09 AM Andrey Gura  wrote:
> > >
> > >> Igor,
> > >>
> > >> Thanks for the proposal.
> > >>
> > >> I understand that such a situation is almost impossible but "if
> > >> anything can go wrong, it will". Does the protocol take into account
> > >> that different connections on different client instances theoretically
> > >> could generate an already existing connection ID?
> > >>
> > >> Also, do I understand correctly that a server has enough information
> > >> about client connections so it will be possible to observe a
> > >> connections list on the server? It would be useful for cluster
> > >> monitoring purposes.
> > >>
> > >> On Tue, May 17, 2022 at 3:11 PM Igor Sapego  wrote:
> > >> >
> > >> > 1. I've tried to clarify IDs part;
> > >> > 2. Maybe you are right, though in this case we'd need to use
> > >> authentication
> > >> > in ConnectionRestoreReq. Which sounds reasonable to me.
> > >> >
> > >> > Best Regards,
> > >> > Igor
> > >> >
> > >> >
> > >> > On Tue, May 17, 2022 at 10:47 AM Pavel Tupitsyn  > >
> > >> > wrote:
> > >> >
> > >> > > Igor,
> > >> > >
> > >> > > The proposal looks good to me. Very detailed!
> > >> > >
> > >> > > A couple comments:
> > >> > >
> > >> > > 1. There is a bit of a term mixup with "Connection ID", "Node ID",
> > >> "Token"
> > >> > > - can you please review those?
> > >> > >
> > >> > > 2. > The Connection ID should be generated using a proper secure
> > >> algorithm
> > >> > > (additional research is required here) to make sure an intruder can
> > >> not
> > >> > > generate an existing Connection ID
> > >> > > Not sure about the reasoning here. I think randomUUID() should be
> > >> enough:
> > >> > > - In the case of an unsecured cluster it does not matter, because
> > >> anyone
> > >> > > can do anything.
> > >> > > - In the case of a secured cluster it does not matter, because
> > >> > > authentication/authorization keeps intruders out.
> > >> > >
> > >> > >
> > >> > > On Mon, May 16, 2022 at 11:07 PM Igor Sapego 
> > >> wrote:
> > >> > >
> > >> > > > Hi, Igniters
> > >> > > >
> > >> > > > I've prepared an IEP for Ignite 3 Client Lifecycle [1]. The main
> > >> idea is
> > >> > > to
> > >> > > > define client lifecycle as well as core algorithms and mechanisms
> > >> used by
> > >> > > > clients. This proposal can be used as a reference for
> > >> implementation of a
> > >> > > > new client for Ignite when dealing with such problems as:
> > >> > > >
> > >> > > >- Resolving of user-provided addresses;
> > >> > > >- Initial connection to a cluster;
> > >> > > >- Maintaining cluster connection;
> > >> > > >- Connection recovery;
> > >> > > >- Connection break handling.
> > >> > > >
> > >> > > > So take a look and let me know 

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Alexander Polovtcev]

> I get your concerns about additional library, but
micronaut itself is quite small.

I'm not that scared of having a big library, I just believe that it might
be possible to avoid using it altogether. Don't swagger generators have
their own annotations?

> The IEP replaces (that is
important) one with another and I’ll add this
information into the description.

My concern is that we will still use Jackson in other modules, like
sql-engine, for example. Or will you replace this library completely in all
modules?

On Fri, May 20, 2022 at 3:39 PM Aleksandr Pakhomov  wrote:

> Hi Andrey,
>
> Thank you for the valuable arguments.
>
> Speaking about micronaut, it is a popular library that
> provides a lot of build-in features like error handling,
> auth, IoC, test infrastructure, and many more. The main
> functionality of micronaut framework is REST, so this
> library is scanned for vulnerabilities regularly and fixes
> them asap (looking to [1] it takes about a week).
> I don't  think that Ignite REST server implementation
> will be scanned as regular as micronaut and issues
> will be fixed as quickly as micronaut does.
>
> As for autogenerated spec, I would mention that manual
> spec writing introduces the second source of truth for
> the API. So, implementation declares one API, spec
> declares another API and there is no prooved contract
> between them. For example, a developer adds "name"
> parameter to the existing endpoint and goes to the
> spec and adds "names" parameter there (makes a mistake).
> What is the right parameter here "name" or "names"?
> Also, if there won't be a compatibility test this mistake could
> go to the production and API spec consumers could get
> a REST client that is not compatible with the server.
>
>
> > On 19 May 2022, at 00:32, Andrey Gura  wrote:
> >
> > I personally don't support any additional 3rd party dependencies as
> > well as I don't fully understand the value of autogenerated specs for
> > REST endpoints. In my opinion we have another option: writing spec
> > manually. This option doesn't require any of proposed dependencies and
> > allow to avoid possible vulnerabilities. Of course at the cost of
> > manual actions.
> >
> > I understand that my statement is arguable. So I'll just wait for
> > opinions of other community members.
>
>

-- 
With regards,
Aleksandr Polovtcev

A new feedback has been added : 78

[Bugyard]

A new feedback has been added, go to bugyard.io to see all the details...

https://bugyard.io

A new feedback has been added 

"This section is empty, which is pretty unhelpful."   by bgardner 

View feedback 
https://app.bugyard.io/web/app/rycqZJDyY/f/62899d2f5db427001462d201

restful api get key result is null,but java thin clien get key has result

[wkhapy...@163.com]

hello:
i use java thin client insert a class type key
like below
public class ContactKey {
private Long id;}and i put contactKey in ignite/libs dir.when i use java 
thin client call get has result.but when i use restful api call get ,it result 
is null.i use restful scan it also has result.how can i fix issue


wkhapy...@163.com

restful api get key result is null,but java thin clien get key has result

[wkhapy...@163.com]

i use java thin client i



wkhapy...@163.com

[MTCGA]: new failures in builds [6442712] needs to be handled

[ignitetcbot]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *Test with high flaky rate in master 
IgnitePdsContinuousRestartTestWithExpiryPolicy.testRebalancingDuringLoad_1000_2_8_1
 
https://ci2.ignite.apache.org/test/-1145411797873198509?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E
 Changes may lead to failure were done by 
 - taras ledkov  
https://ci2.ignite.apache.org/viewModification.html?modId=953933
 - semyon danilov  
https://ci2.ignite.apache.org/viewModification.html?modId=953882

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 12:08:21 23-05-2022 

Re: restful api get key result is null,but java thin clien get key has result

[Ilya Kasnacheev]

Hello!

Please write to user list (u...@ignite.apache.org)

Having said that, I think you need to qualify this key class with package
name when doing REST.

Regards,
-- 
Ilya Kasnacheev


пн, 23 мая 2022 г. в 13:59, wkhapy...@163.com :

> i use java thin client i
>
>
>
> wkhapy...@163.com
>

Re: restful api get key result is null,but java thin clien get key has result

[Николай Ижиков]

Hello.

Please, make sure you have equal value of BinaryConfiguration#compactFooter 
both for server node and for thin client.
Set specific value (true) in both configuration
They are not equals, by default for the backward compatibility reasons.

Having different compactFooter value can lead to described behavior. 

> 23 мая 2022 г., в 15:16, Ilya Kasnacheev  
> написал(а):
> 
> Hello!
> 
> Please write to user list (u...@ignite.apache.org)
> 
> Having said that, I think you need to qualify this key class with package
> name when doing REST.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> пн, 23 мая 2022 г. в 13:59, wkhapy...@163.com :
> 
>> i use java thin client i
>> 
>> 
>> 
>> wkhapy...@163.com
>> 

Apache Ignite 3.0.0 alpha 5 RELEASE [Time, Scope, Manager]

[Andrey Gura]

Hi Igniters,

Four months have passed already since the Ignite 3 alpha 4 release. At
the moment we have a set of features that can be released in order to
give to a user an ability to try the features and share some feedback
with the community. The expected feature list consists of:

  - Pluggable storages: ability to choose a specific storage for a
table (LSM based storage, Page memory persistent and in-memory
storage) with some known limitations.
  - Compute API (A simple remote job execution): The first phase of
Compute API design and implementation. Of course, with known
limitations.
  - Data colocation: The colocation key concept replaces the affinity
key concept. DDL introduces COLOCATE BY clause. Colocated job
execution.
  - Open API for the Ignite REST endpoints: A Specification to
generate a client for any language + auto-generated docs for REST API.
  - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improves UX.
  - Cluster lifecycle: It introduces cluster initialization logic and
allows to specify cluster management and meta storage groups. Improved
node join protocol.
  - Local and distributed recovery: Now it is possible to restart a
cluster/node without data loss.
  - Data rebalance improvements (in progress and could be excluded
from the release), including dynamically changing the number of
partition replicas.
  - Robust client connection with seamless reconnection support and
retry policies.
  - Java API for SQL: A simplified API (design only) for executing SQL
queries on a cluster.

I want to propose myself to be the release manager of the Ignite 3 alpha 5.

Also I propose the following milestones for the release:

Scope Freeze: June 1, 2022
Code Freeze: June 3, 2022
Voting Date: June 6, 2022
Release Date: June 10, 2012

Please, take into account that the proposed release is still alpha, so
we can afford to have such a compressed schedule.

WDYT?

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Aleksandr Pakhomov]

Yes, swagger-core has its own set of annotations [1] 

[1] 
https://github.com/swagger-api/swagger-core/wiki/Swagger-2.X---Annotations#quick-annotation-overview

> On 23 May 2022, at 12:37, Alexander Polovtcev  wrote:
> 
> I'm not that scared of having a big library, I just believe that it might
> be possible to avoid using it altogether. Don't swagger generators have
> their own annotations?

Re: [DISCUSSION] Add additional 3rd party libraries to Ignite 3 Code style and practices

[Aleksandr Pakhomov]

Yes, you are right about the usage of Jackson
in another modules. So, the “replacement” is 
not an argument any more.

> On 23 May 2022, at 12:37, Alexander Polovtcev  wrote:
> 
> My concern is that we will still use Jackson in other modules, like
> sql-engine, for example. Or will you replace this library completely in all
> modules?

[VOTE] Add swagger dependency to Ignite 3

[Aleksandr Pakhomov]

Dear community,

Discussion about 3rd party dependencies took place 
and I think it is time to vote if we agreed to include 
swagger dependency to the Ignite 3 or not. 

The exact list of dependencies could be fined in IEP-87 [1] 
(swagger-annotations, swagger-core, 
swagger-codegen-maven-plugin)

Micronaut is out of the scope of this voting. I will launch 
a separate one.

The vote is formal, see voting guidelines [2]

+1 - to accept additional dependencies to be included to Java code Guidelines 
[3]
0 - don't care either way 
-1 - DO NOT accept (explain why)

[1] 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
[2] https://www.apache.org/foundation/voting.html 
 
[3] 
https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries

[VOTE] Add micronaut dependency to Ignite 3

[Aleksandr Pakhomov]

Dear community,

Micronaut-based REST server implementation was a hot 
topic we discussed in the previous week. So, I've separeted
votes about swagger and micronaut. This vote is about 
adding micronaut to the Ignite 3.

The exact list of dependencies could be fined in IEP-87 [1]
io.micronaut.serde:micronaut-serde
io.micronaut:micronaut-context
io.micronaut:micronaut-http
io.micronaut:micronaut-inject
io.micronaut:micronaut-http-server
io.micronaut:micronaut-runtime
io.micronaut:micronaut-core
io.micronaut:micronaut-http-server-netty
io.micronaut:micronaut-http-netty
io.micronaut:micronaut-buffer-netty
io.micronaut:micronaut-aop
io.micronaut:micronaut-core-reactive
Io.micronaut:micronaut-json-core
io.micronaut:micronaut-jackson-core

Swagger is out of the scope of this voting.

The vote is formal, see voting guidelines [2]

+1 - to accept additional dependencies to be included to Java code Guidelines 
[3]
0 - don't care either way 
-1 - DO NOT accept (explain why)

This vote will be open for at least 4 days till Fri May 27, 2022, 
21:00 Moscow TZ.

[1] 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
[2] https://www.apache.org/foundation/voting.html 
 
[3] 
https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
 

[4] 
https://www.timeanddate.com/countdown/generic?iso=20220527T21&p0=166&msg=%5BVOTE%5D+Add+micronaut+dependency+to+Ignite+3&font=cursive

[VOTE] Add swagger dependency to Ignite 3

[Aleksandr Pakhomov]

This vote will be open for at least 4 days till Fri May 27, 2022, 
21:00 Moscow TZ.

https://www.timeanddate.com/countdown/generic?iso=20220527T21&p0=166&msg=%5BVOTE%5D+Add+swagger+dependency+to+Ignite+3&font=cursive
 

 


> On 23 May 2022, at 19:56, Aleksandr Pakhomov  wrote:
> 
> Dear community,
> 
> Discussion about 3rd party dependencies took place 
> and I think it is time to vote if we agreed to include 
> swagger dependency to the Ignite 3 or not. 
> 
> The exact list of dependencies could be fined in IEP-87 [1] 
> (swagger-annotations, swagger-core, 
> swagger-codegen-maven-plugin)
> 
> Micronaut is out of the scope of this voting. I will launch 
> a separate one.
> 
> The vote is formal, see voting guidelines [2]
> 
> +1 - to accept additional dependencies to be included to Java code Guidelines 
> [3]
> 0 - don't care either way 
> -1 - DO NOT accept (explain why)
> 
> [1] 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
>  
> 
> [2] https://www.apache.org/foundation/voting.html 
>  
> [3] 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries
>  
> 

Re: Apache Ignite 3.0.0 alpha 5 RELEASE [Time, Scope, Manager]

[Юрий]

Hi Andrey,

It's good news, thanks for take RM role and provide list of changes for the
alpha.
Proposed dates looks good to me as well as the scope of the release

пн, 23 мая 2022 г. в 16:05, Andrey Gura :

> Hi Igniters,
>
> Four months have passed already since the Ignite 3 alpha 4 release. At
> the moment we have a set of features that can be released in order to
> give to a user an ability to try the features and share some feedback
> with the community. The expected feature list consists of:
>
>   - Pluggable storages: ability to choose a specific storage for a
> table (LSM based storage, Page memory persistent and in-memory
> storage) with some known limitations.
>   - Compute API (A simple remote job execution): The first phase of
> Compute API design and implementation. Of course, with known
> limitations.
>   - Data colocation: The colocation key concept replaces the affinity
> key concept. DDL introduces COLOCATE BY clause. Colocated job
> execution.
>   - Open API for the Ignite REST endpoints: A Specification to
> generate a client for any language + auto-generated docs for REST API.
>   - Ignite REPL: The Ignite CLI as a REPL with autocompletion and improves
> UX.
>   - Cluster lifecycle: It introduces cluster initialization logic and
> allows to specify cluster management and meta storage groups. Improved
> node join protocol.
>   - Local and distributed recovery: Now it is possible to restart a
> cluster/node without data loss.
>   - Data rebalance improvements (in progress and could be excluded
> from the release), including dynamically changing the number of
> partition replicas.
>   - Robust client connection with seamless reconnection support and
> retry policies.
>   - Java API for SQL: A simplified API (design only) for executing SQL
> queries on a cluster.
>
> I want to propose myself to be the release manager of the Ignite 3 alpha 5.
>
> Also I propose the following milestones for the release:
>
> Scope Freeze: June 1, 2022
> Code Freeze: June 3, 2022
> Voting Date: June 6, 2022
> Release Date: June 10, 2012
>
> Please, take into account that the proposed release is still alpha, so
> we can afford to have such a compressed schedule.
>
> WDYT?
>


-- 
Живи с улыбкой! :D

Re: [VOTE] Add swagger dependency to Ignite 3

[Ilya Kasnacheev]

Hello!

Back when I looked at it, Swagger was very primitive, such as not
supporting primitive types in generated models
https://stackoverflow.com/a/45053804/36498

I'm not sure it is the right tool, please clarify why it is needed.

-0.5 from me (binding)

Regards,
-- 
Ilya Kasnacheev


пн, 23 мая 2022 г. в 19:57, Aleksandr Pakhomov :

> Dear community,
>
> Discussion about 3rd party dependencies took place
> and I think it is time to vote if we agreed to include
> swagger dependency to the Ignite 3 or not.
>
> The exact list of dependencies could be fined in IEP-87 [1]
> (swagger-annotations, swagger-core,
> swagger-codegen-maven-plugin)
>
> Micronaut is out of the scope of this voting. I will launch
> a separate one.
>
> The vote is formal, see voting guidelines [2]
>
> +1 - to accept additional dependencies to be included to Java code
> Guidelines [3]
> 0 - don't care either way
> -1 - DO NOT accept (explain why)
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-87%3A+Open+API+support+for+REST#IEP87:OpenAPIsupportforREST-Additionaldependencies
> [2] https://www.apache.org/foundation/voting.html <
> https://www.apache.org/foundation/voting.html>
> [3]
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2Using3rdpartylibraries

[DISCUSSION] Ignite 3 CLI name

[Aleksandr Pakhomov]

Hi Igniters,

I would like to discuss a name for Ignite 3 CLI [1]. 
Here are candidates from top to bottom.

"Ignite CLI". The most general and straightforward.

"Ignite Shell". It sounds good, only one point: "shell"
is about interpretation, and interaction, but this Ignite CLI 
is not only about interpretation.

"Ignite REPL". Same doubts as for shell.

My personal offer is to use some abstract name that 
does not bring the context about the way how 
commands are executed. It could be "Ignition" and
the terminal application will be called with two
letters ("ignition" too long): "ig", for example: "ig config show".

What do you think? Any ideas are welcome. 

[1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool

[MTCGA]: new failures in builds [6443925] needs to be handled

[ignitetcbot]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 *Test with high flaky rate in master 
TcpDiscoverySslTrustedSelfTest.testDiscoveryEventsDiscard 
https://ci2.ignite.apache.org/test/3386094243324005637?currentProjectId=IgniteTests24Java8&branch=%3Cdefault%3E
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 22:53:20 23-05-2022