[jira] [Created] (IGNITE-14587) ServerImpl.resolveCoordinator returns daemon node as coordinator when daemon node is older then real coordinator

[Rodion (Jira)]

Rodion created IGNITE-14587:
---

 Summary: ServerImpl.resolveCoordinator returns daemon node as 
coordinator when daemon node is older then real coordinator
 Key: IGNITE-14587
 URL: https://issues.apache.org/jira/browse/IGNITE-14587
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.9.1
Reporter: Rodion
Assignee: Rodion
 Fix For: 2.9.2
 Attachments: 
ServerImpl_resolveCoordinator_does_not_work_correctly_with_daemon_nodes.patch

Scenario:
 * Start first server node.
 * Start daemon node
 * Start second server node
 * Stop first server node
 * Check coordinator (ServerImpl#resolveCoordinator and 
ExchangeLatchManager#getLatchCoordinator)

ServerImpl#resolveCoordinator returns daemon node, 
ExchangeLatchManager#getLatchCoordinator - second server node.

Patch with this scenario:

[^ServerImpl_resolveCoordinator_does_not_work_correctly_with_daemon_nodes.patch]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14588) Calcite integration: Wrong processing of nested aggregates

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-14588:
--

 Summary: Calcite integration: Wrong processing of nested aggregates
 Key: IGNITE-14588
 URL: https://issues.apache.org/jira/browse/IGNITE-14588
 Project: Ignite
  Issue Type: Bug
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


The wrong plan is created when nested aggregates are used. 
For example, this query: 
{{SELECT avg(salary) FROM (SELECT avg(salary) as salary FROM employer UNION ALL 
SELECT salary FROM employer)}}
Generates such a plan:
{noformat}
IgniteReduceHashAggregate(group=[{}], AVG(SALARY)=[AVG($0)])
  IgniteExchange(distribution=[single])
IgniteMapHashAggregate(group=[{}], AVG(SALARY)=[AVG($0)])
  IgniteUnionAll(all=[true])
IgniteSingleHashAggregate(group=[{}], SALARY=[AVG($0)])
  IgniteIndexScan(table=[[PUBLIC, EMPLOYER]], index=[_key_PK], 
requiredColumns=[{3}])
IgniteIndexScan(table=[[PUBLIC, EMPLOYER]], index=[_key_PK], 
requiredColumns=[{3}])
{noformat}

With this plan, in subquery data is aggregated locally on nodes and can produce 
the wrong results.
For example:
{code:java}
@Test
public void aggregateNested() throws Exception {
String cacheName = "employer";

IgniteCache employer = client.getOrCreateCache(new 
CacheConfiguration()
.setName(cacheName)
.setSqlSchema("PUBLIC")
.setIndexedTypes(Integer.class, Employer.class)
.setBackups(2)
);

awaitPartitionMapExchange(true, true, null);

List keysNode0 = primaryKeys(grid(0).cache(cacheName), 2);
List keysNode1 = primaryKeys(grid(1).cache(cacheName), 1);

employer.putAll(ImmutableMap.of(
keysNode0.get(0), new Employer("Igor", 1d),
keysNode0.get(1), new Employer("Roman", 2d) ,
keysNode1.get(0), new Employer("Nikolay", 3d)
));

QueryEngine engine = Commons.lookupComponent(grid(1).context(), 
QueryEngine.class);

List>> qry = engine.query(null, "PUBLIC",
"SELECT avg(salary) FROM " +
"(SELECT avg(salary) as salary FROM employer UNION ALL SELECT 
salary FROM employer)");

assertEquals(1, qry.size());

List> rows = qry.get(0).getAll();
assertEquals(1, rows.size());
assertEquals(2d, F.first(F.first(rows)));
}
{code}

With this reproducer we should get 2 as a result (avg(1, 2, 3) = 2, avg(2, 1, 
2, 3) = 2), but actual result is 2.1 (avg(1, 2) = 1.5, avg (3) = 3, avg(1.5, 3, 
1, 2, 3) = 2.1).

Root cause: default {{passThroughDistribution}} is not suitable for "reduce 
aggregate" and "single aggregate" nodes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release pyignite 0.4.0-rc1

[Anton Vinogradov]

Checked the installation from sources.
+1

On Sat, Apr 17, 2021 at 7:47 PM Ivan Daschinsky 
wrote:

> Another reason why whe should host docs on readthedocs.io only is the
> fact,
> that
> pyignite has a separate release cycle. We will release and add more
> features frequently. It's strange to wait for new major ignite version to
> update documentation, as for me.
>
> сб, 17 апр. 2021 г. в 19:36, Ivan Daschinsky :
>
> > Denis, I don't think so. It's maybe sound strange, but python community
> > rely on readthedocs.io and it already has all neede doc's updated.
> > Python package already has comprehensive documentation in the rst format
> > and it is hosted and automatically built on every push.
> > Here is the docs of rc1, that is built on git tag
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1/
> > When we release, I simply just add tag on commit and fresh brand new docs
> > will be available.
> > Link to this doc will be in pypi.org index.
> >
> > I suppose, that the best option is just add a link to readthedocs.io on
> > https://ignite.apache.org/doc and just completely remove all other
> python
> > stuff from it.
> > It is strange to support and edit both versions, as for me.
> >
> > сб, 17 апр. 2021 г. в 19:14, Denis Magda :
> >
> >> +1
> >>
> >> Does it require documentation updates?
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Fri, Apr 16, 2021 at 10:05 AM Ivan Daschinsky 
> >> wrote:
> >>
> >> > Ivan Daschinsky 
> >> > чт, 15 апр., 21:37 (19 часов назад)
> >> > кому: dev
> >> > Dear Igniters!
> >> >
> >> > Release candidate binaries are at least uploaded and ready for vote
> >> > You can find them here:
> >> > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.4.0-rc1
> >> >
> >> > If you follow the link above, you will find source package (*.tar.gz
> and
> >> > *.zip)
> >> > and binary packages (wheels) for windows (x86, amd64) and linux (x86
> and
> >> > x86_64)
> >> > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> >> >
> >> > You can install binary package for specific version of python using
> pip
> >> > For example do this on linux for python 3.8
> >> > >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
> >> >
> >> > You can build and install package from source using this command:
> >> > >> pip install pyignite-0.4.0.tar.gz
> >> > You can build wheel on your platform using this command:
> >> > >> pip wheel --no-deps pyignite-0.4.0.tar.gz
> >> >
> >> > For building C module, you should have python headers and C compiler
> >> > installed.
> >> > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> >> > In Mac OS X xcode-tools and python from homebrew are the best option.
> >> >
> >> > In order to check whether C module works, use following:
> >> > >> from pyignite import _cutils
> >> > >> print(_cutils.hashcode('test'))
> >> > >> 3556498
> >> >
> >> > You can find documentation here:
> >> >
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1
> >> >
> >> > You can find examples here (to check them, you should start ignite
> >> > locally):
> >> >
> >> >
> >>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1/examples.html
> >> > Also, examples can be found in source archive in examples subfolder.
> >> > docker-compose.yml is supplied in order to start ignite quickly.
> >> >
> >> > Release notes:
> >> >
> >> >
> >>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9fee8ead3f70718767f6e11f93d1c7e77c61657b;hb=466b54527e6e42bc585c594d840a959d0b8626ef
> >> >
> >> > Git release tag was created:
> >> >
> >> >
> >>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.4.0.rc1
> >> >
> >> > The vote is formal, see voting guidelines
> >> > https://www.apache.org/foundation/voting.html
> >> >
> >> > +1 - to accept pyignite-0.4.0-rc1
> >> > 0 - don't care either way
> >> > -1 - DO NOT accept pyignite-0.4.0-rc1
> >> >
> >> > The vote will end at April, 21 15:00 UTC.
> >> >
> >>
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>

Re: Building Ignite with Adopt OpenJDK 11

[Marius Filip]

Thank you, Vishwas

Can you point to the reference in the documentation for this information?

If it is missing (and the info is transmitted via email) then I'll raise a
ticket and add it myself.

Thanks,
Marius Filip

On Mon, Apr 19, 2021 at 3:41 AM Vishwas Bm  wrote:

> Hi Marius,
>
> This is more of a mvn repository issue. The jacorb jar is found at below
> link
>
>
> https://repository.ow2.org/nexus/content/repositories/ow2-legacy/org/jacorb/jacorb/2.2.3-jonas-patch-20071018/
>
> You need to add this repo in your settings.xml file.
>
>
> Regards,
> Vishwas
>
>
> On Sun, 18 Apr, 2021, 23:51 Marius Filip, 
> wrote:
>
>> Hi
>>
>> I followed the Git Workflow instructions for contributors at the bottom
>> of:
>>
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>>
>> and then the build instructions in DEVNOTES.txt:
>>
>> mvn clean install -Pall-java,all-scala,licenses -DskipTests
>>
>> I get the following error:
>>
>> [ERROR] Failed to execute goal on project ignite-jta: Could not resolve
>> dependencies for project org.apache.ignite:ignite-jta:jar:2.11.0-SNAPSHOT:
>> Failed to collect dependencies at org.ow2.jotm:jotm-core:jar:2.2.3 ->
>> org.ow2.carol:carol:jar:3.0.8 ->
>> org.jacorb:jacorb:jar:2.2.3-jonas-patch-20071018: Failed to read artifact
>> descriptor for org.jacorb:jacorb:jar:2.2.3-jonas-patch-20071018: Could not
>> transfer artifact org.jacorb:jacorb:pom:2.2.3-jonas-patch-20071018 from/to
>> maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for
>> repositories: [redhat-ga-repository (
>> http://maven.repository.redhat.com/ga/,
>> default, releases), apache.snapshots (
>> http://repository.apache.org/snapshots,
>> default, snapshots), ow2-snapshot (
>> http://repository.ow2.org/nexus/content/repositories/snapshots, default,
>> snapshots), ow2 (http://maven.ow2.org/maven2, default, releases)] ->
>> [Help
>> 1]
>>
>> Is this related to using JDK 11 to build?
>>
>> If this is a known issue or Java 11 is not supported, any pointer to the
>> correct info is much appreciated.
>>
>> Thanks,
>> Marius Filip
>>
>

[jira] [Created] (IGNITE-14589) Calcite engine. Numerous problem with type Decimal

[Konstantin Orlov (Jira)]

Konstantin Orlov created IGNITE-14589:
-

 Summary: Calcite engine. Numerous problem with type Decimal
 Key: IGNITE-14589
 URL: https://issues.apache.org/jira/browse/IGNITE-14589
 Project: Ignite
  Issue Type: Bug
  Components: sql
Reporter: Konstantin Orlov


There are numerous problem with a decimal types:
* very big numbers printed in scientific notation
* leading and trailing zeros are truncated when converting to string
* casting to precise scale is not working( {{select cast('0.01' as decimal(10, 
1)) * 10}} returns 0.1 instead of 0)

Affected tests:
{{src/test/sql/types/decimal}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14590) Calcite engine. Sort out the types/decimal and types/string directories

[Konstantin Orlov (Jira)]

Konstantin Orlov created IGNITE-14590:
-

 Summary: Calcite engine. Sort out the types/decimal and 
types/string directories
 Key: IGNITE-14590
 URL: https://issues.apache.org/jira/browse/IGNITE-14590
 Project: Ignite
  Issue Type: Task
  Components: sql
Reporter: Konstantin Orlov
Assignee: Konstantin Orlov


We need to sort out results of the {{ScriptRunnerTestSuite}} to mute an every 
red test and file a ticket for related problems.

This ticket covers only two directories: {{types/decimal}} and {{types/string}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-14591) Add Javadoc rules to maven checkstyle plugin.

[Andrey Mashenkov (Jira)]

Andrey Mashenkov created IGNITE-14591:
-

 Summary: Add Javadoc rules to maven checkstyle plugin.
 Key: IGNITE-14591
 URL: https://issues.apache.org/jira/browse/IGNITE-14591
 Project: Ignite
  Issue Type: Improvement
Reporter: Andrey Mashenkov


Let's add javadoc rules to maven-checkstyle-plugin and fix the Codestyle guide.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release pyignite 0.4.0-rc1

[Anton Vinogradov]

Also checked the hashcode generation
>> from pyignite import _cutils
>> print(_cutils.hashcode('test'))
>> 3556498

On Mon, Apr 19, 2021 at 11:01 AM Anton Vinogradov  wrote:

> Checked the installation from sources.
> +1
>
> On Sat, Apr 17, 2021 at 7:47 PM Ivan Daschinsky 
> wrote:
>
>> Another reason why whe should host docs on readthedocs.io only is the
>> fact,
>> that
>> pyignite has a separate release cycle. We will release and add more
>> features frequently. It's strange to wait for new major ignite version to
>> update documentation, as for me.
>>
>> сб, 17 апр. 2021 г. в 19:36, Ivan Daschinsky :
>>
>> > Denis, I don't think so. It's maybe sound strange, but python community
>> > rely on readthedocs.io and it already has all neede doc's updated.
>> > Python package already has comprehensive documentation in the rst format
>> > and it is hosted and automatically built on every push.
>> > Here is the docs of rc1, that is built on git tag
>> >
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1/
>> > When we release, I simply just add tag on commit and fresh brand new
>> docs
>> > will be available.
>> > Link to this doc will be in pypi.org index.
>> >
>> > I suppose, that the best option is just add a link to readthedocs.io on
>> > https://ignite.apache.org/doc and just completely remove all other
>> python
>> > stuff from it.
>> > It is strange to support and edit both versions, as for me.
>> >
>> > сб, 17 апр. 2021 г. в 19:14, Denis Magda :
>> >
>> >> +1
>> >>
>> >> Does it require documentation updates?
>> >>
>> >> -
>> >> Denis
>> >>
>> >>
>> >> On Fri, Apr 16, 2021 at 10:05 AM Ivan Daschinsky > >
>> >> wrote:
>> >>
>> >> > Ivan Daschinsky 
>> >> > чт, 15 апр., 21:37 (19 часов назад)
>> >> > кому: dev
>> >> > Dear Igniters!
>> >> >
>> >> > Release candidate binaries are at least uploaded and ready for vote
>> >> > You can find them here:
>> >> > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.4.0-rc1
>> >> >
>> >> > If you follow the link above, you will find source package (*.tar.gz
>> and
>> >> > *.zip)
>> >> > and binary packages (wheels) for windows (x86, amd64) and linux (x86
>> and
>> >> > x86_64)
>> >> > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg
>> signatures.
>> >> >
>> >> > You can install binary package for specific version of python using
>> pip
>> >> > For example do this on linux for python 3.8
>> >> > >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
>> >> >
>> >> > You can build and install package from source using this command:
>> >> > >> pip install pyignite-0.4.0.tar.gz
>> >> > You can build wheel on your platform using this command:
>> >> > >> pip wheel --no-deps pyignite-0.4.0.tar.gz
>> >> >
>> >> > For building C module, you should have python headers and C compiler
>> >> > installed.
>> >> > (i.e. for ubuntu sudo apt install build-essential python3-dev)
>> >> > In Mac OS X xcode-tools and python from homebrew are the best option.
>> >> >
>> >> > In order to check whether C module works, use following:
>> >> > >> from pyignite import _cutils
>> >> > >> print(_cutils.hashcode('test'))
>> >> > >> 3556498
>> >> >
>> >> > You can find documentation here:
>> >> >
>> >>
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1
>> >> >
>> >> > You can find examples here (to check them, you should start ignite
>> >> > locally):
>> >> >
>> >> >
>> >>
>> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.4.0.rc1/examples.html
>> >> > Also, examples can be found in source archive in examples subfolder.
>> >> > docker-compose.yml is supplied in order to start ignite quickly.
>> >> >
>> >> > Release notes:
>> >> >
>> >> >
>> >>
>> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9fee8ead3f70718767f6e11f93d1c7e77c61657b;hb=466b54527e6e42bc585c594d840a959d0b8626ef
>> >> >
>> >> > Git release tag was created:
>> >> >
>> >> >
>> >>
>> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.4.0.rc1
>> >> >
>> >> > The vote is formal, see voting guidelines
>> >> > https://www.apache.org/foundation/voting.html
>> >> >
>> >> > +1 - to accept pyignite-0.4.0-rc1
>> >> > 0 - don't care either way
>> >> > -1 - DO NOT accept pyignite-0.4.0-rc1
>> >> >
>> >> > The vote will end at April, 21 15:00 UTC.
>> >> >
>> >>
>> >
>> >
>> > --
>> > Sincerely yours, Ivan Daschinskiy
>> >
>>
>

[DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Andrey Mashenkov]

Hi Igniters,

We use JDK Javadoc tool to validate javadocs on TC (Javadoc suite) in
Ignite 2 and now in Ignite 3.
A javadoc tool is designed for javadoc site generation that also performs
some basic checks and markup validation,
but has nothing to do with javadoc styles.

I've found maven-checkstyle-plugin has modules for javadoc style checking,
which looks more suited for the issue.
I've tried to turn on javadoc checks in checkstyle plugin, then run it
against Ignite-3 main branch and got 1200+ errors.

For Ignite-2 thing may goes worse and numbers can be much higher,
but let please, start a separate discussion for this if one feels it make
sense.

Javadoc is part of documentation which a face of product and we MUST keep
high standards for javadocs as well.

Let's improve this within the ticket [1] regarding the next suggestions:
1. Add Javadoc checks to mavan-checkstyle-plugin. I've made a PR for
Ignite-3 [1] to turn on style checks for javadocs.

2. Keep the current Javadoc TC suite as is. because it allow detecting
markup errors regarding current javadoc tool version capabilities.

3. Fix the Codestyle guidelines to follow higher standards.
3.1. Disallow empty javadocs (or with missed description) for member that
can be used outside the class/package/module by users or other developers.
I believe all methods/classes/fields that can be used by developers
(public, protected, package visible) MUST have meaningful description,
excepts may be tests, well-known constants (e.g. serialVersionUid), and
private members.
Actually, it not a big deal to put few words into javadoc even if the
method looks simple,
if one feels difficulties expressing a class/method purpose, then most
likely refactoring is needed.

3.2. Check all params/throws/returns/generics/deprecates MUST be
well-documented and goes in order.

3.4. Suggest to allow optional tags @apiNote, @implSpec, @implNote as
described in [3],
to put e.g. expectations/requirements of implementation for developers that
may be non-obvious.
The tags values are rendered in separate blocks on javadoc site.

3.5 However, one-liner javadoc like "{@inheritDoc}" does nothing and can be
safely omitted. I'd keep it.

3.6 Add javadoc checks for 'package-info'. Do we want an additional
requirement to every package has package-info?

Working on the patch I've found it is impossible to have different policies
in the same config for different scopes: source and test code.
Thus, we can either exclude tests from style checks at all, which looks
like not a good idea,
or have different configs with different policies for source and test code.

Any thoughts?

[1] https://issues.apache.org/jira/browse/IGNITE-14591
[2] https://github.com/apache/ignite-3/pull/98
[3] http://openjdk.java.net/jeps/8068562

-- 
Best regards,
Andrey V. Mashenkov

Re: Building Ignite with Adopt OpenJDK 11

[vbm]

Hi Marius,

Similar build failure issue was discussed in user mailing list.
http://apache-ignite-users.70518.x6.nabble.com/build-failure-for-module-hibernate-5-3-tp33621p33629.html

As suggested in above link, you can add to your settings.xml: 
https://repository.ow2.org/nexus/content/repositories/public/


Regards,
Vishwas



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-14592) Incapsulate config modification logic into callback

[Nikolay Izhikov (Jira)]

Nikolay Izhikov created IGNITE-14592:


 Summary: Incapsulate config modification logic into callback
 Key: IGNITE-14592
 URL: https://issues.apache.org/jira/browse/IGNITE-14592
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov


Right now extensions of {{IgniteSpec}} can't provide custom config files and 
it's templates because only template returned from spec if 
{{IgniteConfigTemplate}}.

We need to return array of tuples from {{IgniteSpec#config_templates}} that 
specify config files required to be generated to run Ignite based node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release pyignite 0.4.0-rc1

[Surkov]

Tested signatures, tested hashsums, tested installation on Mac Os X 10.15.7
on python 3.9.4.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Ivan Pavlukhin]

Hi Andrey and Igniters,

Sorry if I misread something but I have totally different opinion in
one aspect. In my mind it is much better in practice to follow strict
rules for public API javadocs but not for internals. I would use
static checks for API packages and disable them for internal ones and
refine code readability during code review. Main motivation here is
that ubiquitous javadocs did not work well in ignite-2 and I believe
it would not in ignite-3.

2021-04-19 13:30 GMT+03:00, Andrey Mashenkov :
> Hi Igniters,
>
> We use JDK Javadoc tool to validate javadocs on TC (Javadoc suite) in
> Ignite 2 and now in Ignite 3.
> A javadoc tool is designed for javadoc site generation that also performs
> some basic checks and markup validation,
> but has nothing to do with javadoc styles.
>
> I've found maven-checkstyle-plugin has modules for javadoc style checking,
> which looks more suited for the issue.
> I've tried to turn on javadoc checks in checkstyle plugin, then run it
> against Ignite-3 main branch and got 1200+ errors.
>
> For Ignite-2 thing may goes worse and numbers can be much higher,
> but let please, start a separate discussion for this if one feels it make
> sense.
>
> Javadoc is part of documentation which a face of product and we MUST keep
> high standards for javadocs as well.
>
> Let's improve this within the ticket [1] regarding the next suggestions:
> 1. Add Javadoc checks to mavan-checkstyle-plugin. I've made a PR for
> Ignite-3 [1] to turn on style checks for javadocs.
>
> 2. Keep the current Javadoc TC suite as is. because it allow detecting
> markup errors regarding current javadoc tool version capabilities.
>
> 3. Fix the Codestyle guidelines to follow higher standards.
> 3.1. Disallow empty javadocs (or with missed description) for member that
> can be used outside the class/package/module by users or other developers.
> I believe all methods/classes/fields that can be used by developers
> (public, protected, package visible) MUST have meaningful description,
> excepts may be tests, well-known constants (e.g. serialVersionUid), and
> private members.
> Actually, it not a big deal to put few words into javadoc even if the
> method looks simple,
> if one feels difficulties expressing a class/method purpose, then most
> likely refactoring is needed.
>
> 3.2. Check all params/throws/returns/generics/deprecates MUST be
> well-documented and goes in order.
>
> 3.4. Suggest to allow optional tags @apiNote, @implSpec, @implNote as
> described in [3],
> to put e.g. expectations/requirements of implementation for developers that
> may be non-obvious.
> The tags values are rendered in separate blocks on javadoc site.
>
> 3.5 However, one-liner javadoc like "{@inheritDoc}" does nothing and can be
> safely omitted. I'd keep it.
>
> 3.6 Add javadoc checks for 'package-info'. Do we want an additional
> requirement to every package has package-info?
>
> Working on the patch I've found it is impossible to have different policies
> in the same config for different scopes: source and test code.
> Thus, we can either exclude tests from style checks at all, which looks
> like not a good idea,
> or have different configs with different policies for source and test code.
>
> Any thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-14591
> [2] https://github.com/apache/ignite-3/pull/98
> [3] http://openjdk.java.net/jeps/8068562
>
> --
> Best regards,
> Andrey V. Mashenkov
>


-- 

Best regards,
Ivan Pavlukhin

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Alexey Kukushkin]

I personally do not understand why we need mandatory javadoc even in public
modules. I think javadoc is needed only when the code is not
self-descriptive. What value does a javadoc like this bring

to a developer:

/** Default metrics history size (value is {@code 1}). */
public static final int DFLT_METRICS_HISTORY_SIZE = 1;

/** Metrics history time. */
private int metricsHistSize = DFLT_METRICS_HISTORY_SIZE;

BTW, I picked a random example and it already has a copy-paste mistake in
the javadoc, which is there for years. I think that indicates it has no
real value and developers just do it to comply with the rule.

Some say mandatory javadoc is for the discipline but I think Apache Ignite
developers are mature enough to understand when additional documentation is
really required.



пн, 19 апр. 2021 г. в 16:37, Ivan Pavlukhin :

> Hi Andrey and Igniters,
>
> Sorry if I misread something but I have totally different opinion in
> one aspect. In my mind it is much better in practice to follow strict
> rules for public API javadocs but not for internals. I would use
> static checks for API packages and disable them for internal ones and
> refine code readability during code review. Main motivation here is
> that ubiquitous javadocs did not work well in ignite-2 and I believe
> it would not in ignite-3.
>
> 2021-04-19 13:30 GMT+03:00, Andrey Mashenkov :
> > Hi Igniters,
> >
> > We use JDK Javadoc tool to validate javadocs on TC (Javadoc suite) in
> > Ignite 2 and now in Ignite 3.
> > A javadoc tool is designed for javadoc site generation that also performs
> > some basic checks and markup validation,
> > but has nothing to do with javadoc styles.
> >
> > I've found maven-checkstyle-plugin has modules for javadoc style
> checking,
> > which looks more suited for the issue.
> > I've tried to turn on javadoc checks in checkstyle plugin, then run it
> > against Ignite-3 main branch and got 1200+ errors.
> >
> > For Ignite-2 thing may goes worse and numbers can be much higher,
> > but let please, start a separate discussion for this if one feels it make
> > sense.
> >
> > Javadoc is part of documentation which a face of product and we MUST keep
> > high standards for javadocs as well.
> >
> > Let's improve this within the ticket [1] regarding the next suggestions:
> > 1. Add Javadoc checks to mavan-checkstyle-plugin. I've made a PR for
> > Ignite-3 [1] to turn on style checks for javadocs.
> >
> > 2. Keep the current Javadoc TC suite as is. because it allow detecting
> > markup errors regarding current javadoc tool version capabilities.
> >
> > 3. Fix the Codestyle guidelines to follow higher standards.
> > 3.1. Disallow empty javadocs (or with missed description) for member that
> > can be used outside the class/package/module by users or other
> developers.
> > I believe all methods/classes/fields that can be used by developers
> > (public, protected, package visible) MUST have meaningful description,
> > excepts may be tests, well-known constants (e.g. serialVersionUid), and
> > private members.
> > Actually, it not a big deal to put few words into javadoc even if the
> > method looks simple,
> > if one feels difficulties expressing a class/method purpose, then most
> > likely refactoring is needed.
> >
> > 3.2. Check all params/throws/returns/generics/deprecates MUST be
> > well-documented and goes in order.
> >
> > 3.4. Suggest to allow optional tags @apiNote, @implSpec, @implNote as
> > described in [3],
> > to put e.g. expectations/requirements of implementation for developers
> that
> > may be non-obvious.
> > The tags values are rendered in separate blocks on javadoc site.
> >
> > 3.5 However, one-liner javadoc like "{@inheritDoc}" does nothing and can
> be
> > safely omitted. I'd keep it.
> >
> > 3.6 Add javadoc checks for 'package-info'. Do we want an additional
> > requirement to every package has package-info?
> >
> > Working on the patch I've found it is impossible to have different
> policies
> > in the same config for different scopes: source and test code.
> > Thus, we can either exclude tests from style checks at all, which looks
> > like not a good idea,
> > or have different configs with different policies for source and test
> code.
> >
> > Any thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-14591
> > [2] https://github.com/apache/ignite-3/pull/98
> > [3] http://openjdk.java.net/jeps/8068562
> >
> > --
> > Best regards,
> > Andrey V. Mashenkov
> >
>
>
> --
>
> Best regards,
> Ivan Pavlukhin
>


-- 
Best regards,
Alexey

[Meetup] Native persistence storage overview. April 27 2021

[Anton,Kalashnikov]

Hi Igniters,

There is meetup next week, where I want to share my knowledge about 
Ignite Persistence module.  It will be high level overview of the main 
components and the main ideas behind the Persistence module. So if you 
are a new ignite contributor or you don't feel  confident enough in 
Ignite persistence it can be helpful for you.


If you interested you can join at 8AM(PST), April 27. More info - 
https://www.meetup.com/ru-RU/Apache-Ignite-Virtual-Meetup/events/277298901.


--
Best regards,
Anton Kalashnikov

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Pavel Tupitsyn]

Agree with Ivan - internal code does not need mandatory Javadoc.
Most of it is meaningless and does not bring any value, just wastes
everyone's time.

Alexey, I think public APIs should always have Javadoc,
even if it is the same thing as the member name, but with spaces -
this will make the documentation nicer and easier to search.

On Mon, Apr 19, 2021 at 5:21 PM Alexey Kukushkin 
wrote:

> I personally do not understand why we need mandatory javadoc even in public
> modules. I think javadoc is needed only when the code is not
> self-descriptive. What value does a javadoc like this bring
> <
> https://github.com/apache/ignite/blob/2.10.0/modules/core/src/main/java/org/apache/ignite/configuration/IgniteConfiguration.java
> >
> to a developer:
>
> /** Default metrics history size (value is {@code 1}). */
> public static final int DFLT_METRICS_HISTORY_SIZE = 1;
>
> /** Metrics history time. */
> private int metricsHistSize = DFLT_METRICS_HISTORY_SIZE;
>
> BTW, I picked a random example and it already has a copy-paste mistake in
> the javadoc, which is there for years. I think that indicates it has no
> real value and developers just do it to comply with the rule.
>
> Some say mandatory javadoc is for the discipline but I think Apache Ignite
> developers are mature enough to understand when additional documentation is
> really required.
>
>
>
> пн, 19 апр. 2021 г. в 16:37, Ivan Pavlukhin :
>
> > Hi Andrey and Igniters,
> >
> > Sorry if I misread something but I have totally different opinion in
> > one aspect. In my mind it is much better in practice to follow strict
> > rules for public API javadocs but not for internals. I would use
> > static checks for API packages and disable them for internal ones and
> > refine code readability during code review. Main motivation here is
> > that ubiquitous javadocs did not work well in ignite-2 and I believe
> > it would not in ignite-3.
> >
> > 2021-04-19 13:30 GMT+03:00, Andrey Mashenkov  >:
> > > Hi Igniters,
> > >
> > > We use JDK Javadoc tool to validate javadocs on TC (Javadoc suite) in
> > > Ignite 2 and now in Ignite 3.
> > > A javadoc tool is designed for javadoc site generation that also
> performs
> > > some basic checks and markup validation,
> > > but has nothing to do with javadoc styles.
> > >
> > > I've found maven-checkstyle-plugin has modules for javadoc style
> > checking,
> > > which looks more suited for the issue.
> > > I've tried to turn on javadoc checks in checkstyle plugin, then run it
> > > against Ignite-3 main branch and got 1200+ errors.
> > >
> > > For Ignite-2 thing may goes worse and numbers can be much higher,
> > > but let please, start a separate discussion for this if one feels it
> make
> > > sense.
> > >
> > > Javadoc is part of documentation which a face of product and we MUST
> keep
> > > high standards for javadocs as well.
> > >
> > > Let's improve this within the ticket [1] regarding the next
> suggestions:
> > > 1. Add Javadoc checks to mavan-checkstyle-plugin. I've made a PR for
> > > Ignite-3 [1] to turn on style checks for javadocs.
> > >
> > > 2. Keep the current Javadoc TC suite as is. because it allow detecting
> > > markup errors regarding current javadoc tool version capabilities.
> > >
> > > 3. Fix the Codestyle guidelines to follow higher standards.
> > > 3.1. Disallow empty javadocs (or with missed description) for member
> that
> > > can be used outside the class/package/module by users or other
> > developers.
> > > I believe all methods/classes/fields that can be used by developers
> > > (public, protected, package visible) MUST have meaningful description,
> > > excepts may be tests, well-known constants (e.g. serialVersionUid), and
> > > private members.
> > > Actually, it not a big deal to put few words into javadoc even if the
> > > method looks simple,
> > > if one feels difficulties expressing a class/method purpose, then most
> > > likely refactoring is needed.
> > >
> > > 3.2. Check all params/throws/returns/generics/deprecates MUST be
> > > well-documented and goes in order.
> > >
> > > 3.4. Suggest to allow optional tags @apiNote, @implSpec, @implNote as
> > > described in [3],
> > > to put e.g. expectations/requirements of implementation for developers
> > that
> > > may be non-obvious.
> > > The tags values are rendered in separate blocks on javadoc site.
> > >
> > > 3.5 However, one-liner javadoc like "{@inheritDoc}" does nothing and
> can
> > be
> > > safely omitted. I'd keep it.
> > >
> > > 3.6 Add javadoc checks for 'package-info'. Do we want an additional
> > > requirement to every package has package-info?
> > >
> > > Working on the patch I've found it is impossible to have different
> > policies
> > > in the same config for different scopes: source and test code.
> > > Thus, we can either exclude tests from style checks at all, which looks
> > > like not a good idea,
> > > or have different configs with different policies for source and test
> > code.
> > >
> > > Any thoughts?
> > >

Re: [DISCUSSION] Javadoc style requirements and checks in Ignite-3.

[Andrey Mashenkov]

Alexey,

These are bad examples and unfortunately, most of the Ignite code doesn't
look self-descriptive.
(Do you feel the difference between @SerializableTransient and
@TransientSerializable?)

To understand the semantic of e.g. 'metricsHistSize' you have to analyze
its usages.
Getter and setter for this property look better, but it still not clear
what happens with metric values above the limit?

I have another example, one of the core components GridDhtPartitionDemander
has totally useless javadoc.
It is obviously has nothing with thread pool + "local cache" phrase looks
very confusing.

/**
 * Thread pool for requesting partitions from other nodes and
populating local cache.
 */
public class GridDhtPartitionDemander

To understand the purpose of this internal component I have to analyze its
code and usages.
However, if one will face unexpected behavior, he/she may be confused if it
is a lack of javadoc or wrong behavior.

There is no way to restrict or avoid such issues with any checks.
Agree, meaningful javadocs as self-descriptive code is more about culture
and discipline.

The absence of javadoc on internal API, unreasonably raises the entry
threshold for new developers and makes the development of intra-component
interaction harder.
I admit a high-level description for public classes may be enough to
resolve this without pushing developers to write empty/useless docs for
every method/field.


On Mon, Apr 19, 2021 at 5:21 PM Alexey Kukushkin 
wrote:

> I personally do not understand why we need mandatory javadoc even in public
> modules. I think javadoc is needed only when the code is not
> self-descriptive. What value does a javadoc like this bring
> <
> https://github.com/apache/ignite/blob/2.10.0/modules/core/src/main/java/org/apache/ignite/configuration/IgniteConfiguration.java
> >
> to a developer:
>
> /** Default metrics history size (value is {@code 1}). */
> public static final int DFLT_METRICS_HISTORY_SIZE = 1;
>
> /** Metrics history time. */
> private int metricsHistSize = DFLT_METRICS_HISTORY_SIZE;
>
> BTW, I picked a random example and it already has a copy-paste mistake in
> the javadoc, which is there for years. I think that indicates it has no
> real value and developers just do it to comply with the rule.
>
> Some say mandatory javadoc is for the discipline but I think Apache Ignite
> developers are mature enough to understand when additional documentation is
> really required.
>
>
>
> пн, 19 апр. 2021 г. в 16:37, Ivan Pavlukhin :
>
> > Hi Andrey and Igniters,
> >
> > Sorry if I misread something but I have totally different opinion in
> > one aspect. In my mind it is much better in practice to follow strict
> > rules for public API javadocs but not for internals. I would use
> > static checks for API packages and disable them for internal ones and
> > refine code readability during code review. Main motivation here is
> > that ubiquitous javadocs did not work well in ignite-2 and I believe
> > it would not in ignite-3.
> >
> > 2021-04-19 13:30 GMT+03:00, Andrey Mashenkov  >:
> > > Hi Igniters,
> > >
> > > We use JDK Javadoc tool to validate javadocs on TC (Javadoc suite) in
> > > Ignite 2 and now in Ignite 3.
> > > A javadoc tool is designed for javadoc site generation that also
> performs
> > > some basic checks and markup validation,
> > > but has nothing to do with javadoc styles.
> > >
> > > I've found maven-checkstyle-plugin has modules for javadoc style
> > checking,
> > > which looks more suited for the issue.
> > > I've tried to turn on javadoc checks in checkstyle plugin, then run it
> > > against Ignite-3 main branch and got 1200+ errors.
> > >
> > > For Ignite-2 thing may goes worse and numbers can be much higher,
> > > but let please, start a separate discussion for this if one feels it
> make
> > > sense.
> > >
> > > Javadoc is part of documentation which a face of product and we MUST
> keep
> > > high standards for javadocs as well.
> > >
> > > Let's improve this within the ticket [1] regarding the next
> suggestions:
> > > 1. Add Javadoc checks to mavan-checkstyle-plugin. I've made a PR for
> > > Ignite-3 [1] to turn on style checks for javadocs.
> > >
> > > 2. Keep the current Javadoc TC suite as is. because it allow detecting
> > > markup errors regarding current javadoc tool version capabilities.
> > >
> > > 3. Fix the Codestyle guidelines to follow higher standards.
> > > 3.1. Disallow empty javadocs (or with missed description) for member
> that
> > > can be used outside the class/package/module by users or other
> > developers.
> > > I believe all methods/classes/fields that can be used by developers
> > > (public, protected, package visible) MUST have meaningful description,
> > > excepts may be tests, well-known constants (e.g. serialVersionUid), and
> > > private members.
> > > Actually, it not a big deal to put few words into javadoc even if the
> > > method looks simple,
> > > if one feels difficulties expressing a class/method purpose, then most
>

Re: Model of permissions for Ignite 3

[Valentin Kulichenko]

Hi folks,

Did you have a discussion? How did it go? Can someone summarize the results?

-Val

On Mon, Apr 12, 2021 at 2:00 AM Kseniya Romanova 
wrote:

> Hi! Scheduled open call for Friday
> https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/277518672/
> Please join to see the zoom link. Consulted with Denis Garus and put the
> topic as "Security", cause it's definitely wider than just permissions.
>
> Cheers!
>
> пн, 12 апр. 2021 г. в 09:25, Alexei Scherbakov <
> alexey.scherbak...@gmail.com
> >:
>
> > +1
> >
> > We should rethink the security model in Ignite 3 and have a default RBAC
> > based implementation, from my point of view.
> > By default, no code should be written to enable and use it.
> >
> > Let's schedule a meeting and discuss the details.
> >
> > вс, 11 апр. 2021 г. в 19:43, Denis Garus :
> >
> > > Andrey, Alexey thank you for the feedback!
> > >
> > > > I suggest decoupling authentication from authorization
> > >
> > > Yes, it would be useful. Existing SecuritySubject and SecurityContext
> > > require reworking.
> > >
> > > > I think it would be great to have a default implementation of
> > > user-role-permission model
> > >
> > > Completely agree it is a cool idea. Ignite should have more default
> > > implementation referred to security.
> > >
> > > > Should we have a community meeting to discuss this?
> > >
> > > Yes, it would be great.
> > > The wish list for Ignite 3 does not contain security improvement that,
> > > IMHO, is wrong.
> > > We should fix that oversight on early-stage Ignite 3 development.
> > >
> > > пт, 9 апр. 2021 г. в 18:47, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> > >
> > > > Hello Denis, Andrey, Igniters,
> > > >
> > > > Why don't we take a step further in improving the security model in
> > > Ignite
> > > > 3? I think it would be great to have a default implementation of
> > > > user-role-permission model in Ignite to be on par with security
> models
> > of
> > > > widely-used databases. This will complement community efforts in
> making
> > > > most of the Ignite 3 behavior to be changeable in runtime.
> > > >
> > > > WDYT? Should we have a community meeting to discuss this?
> > > >
> > > >
> > > > чт, 8 апр. 2021 г. в 23:54, Andrey Kuznetsov :
> > > >
> > > > > Hi Denis!
> > > > >
> > > > > The idea and prototype look great.
> > > > >
> > > > > I'd like to highlight one arguable point. Default authorization
> > > > > implementation still assumes there are permissions provided in
> > > > > SecuritySubject. In turn, authentication is still responsible for
> > > filling
> > > > > these permissions. I suggest decoupling authentication from
> > > > authorization,
> > > > > so that GridSecurityProcessor implementation is fully responsible
> for
> > > > > obtaining permissions for SecuritySubject given on authorization.
> In
> > > > > particular, implementation can choose an existing behavior of
> > bundling
> > > > > permissions with SecuritySubject.
> > > > >
> > > > > Makes sense?
> > > > >
> > > > > чт, 8 апр. 2021 г. в 17:52, Denis Garus :
> > > > >
> > > > > > Sorry, I forgot to point the link
> > > > > >
> > > > > > 1. https://github.com/apache/ignite/pull/8989
> > > > > >
> > > > > > чт, 8 апр. 2021 г. в 17:50, Denis Garus :
> > > > > >
> > > > > > > Hello, Igniters!
> > > > > > >
> > > > > > > I want to propose to improve the way which we use
> > > > > > > to present permissions in Ignite 3.
> > > > > > >
> > > > > > > The model of permission in Ignite has a set of drawbacks.
> > > > > > > The main drawback, IMHO: if you need to add a new permission,
> > > > > > > you should change the core module by extended the
> > > > 'SecurityPermission'
> > > > > > > enum.
> > > > > > > An approach like this becomes more challenged if new permission
> > is
> > > > > > created
> > > > > > > for an extension.
> > > > > > >
> > > > > > > The existing permission model is overcomplicated.
> > > > > > > The SecurityPermission enum is divided into four groups,
> > > > > > > and to determine whether a security subject has been given
> > > > permission,
> > > > > > > a plugin developer has to know what the permission group is.
> > > > > > > But 'CACHE_CREATE' and 'CACHE_DESTROY' are included in two
> groups
> > > > > (system
> > > > > > > operations and cache operations).
> > > > > > > When 'CACHE_CREATE' ('CACHE_DESTROY') is treated as system
> > > > permission,
> > > > > > > it applies to all caches. In other cases, when 'CACHE_CREATE'
> > > > > > > ('CACHE_DESTROY') is treated as cache permission,
> > > > > > > permission checking is executed with the account of the cache
> > name.
> > > > > > > IMHO, this logic is hard to understand.
> > > > > > > There is no ability to represent compound operation as single
> > > > > permission
> > > > > > > and so on.
> > > > > > >
> > > > > > >
> > > > > > > So I would like to suggest using a permission model that is
> based
> > > on
> > > > > > > 'java.security.Permission'.
> > > > > > > I prepared the concept [1] of 

[jira] [Created] (IGNITE-14593) Calcite. Support not equal expression.

[Stanilovsky Evgeny (Jira)]

Stanilovsky Evgeny created IGNITE-14593:
---

 Summary: Calcite. Support not equal expression.
 Key: IGNITE-14593
 URL: https://issues.apache.org/jira/browse/IGNITE-14593
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Stanilovsky Evgeny


Looks like it would be helpful to support "!=" syntax, i.e. :

{code:java}
UPDATE test SET a=7 WHERE id != 3
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)