Re: Move Azure, AWS, GCE to the ignite-extensions
Hi, Maksim. Can you file a ticket about recreating test suites for extensions, please? I will attend to it in nearest time. > On 23 Nov 2021, at 17:14, Maxim Muzafarov wrote: > > Hello Petr, > > Can you assist me with configuring the GCE [1] suite on the TC > Extensions project? Currently, I have an issue with moving environment > variables from the old GCE suite [2] to the new one. > > I need to create the following envs: > - env.test.gce.account.id > - env.test.gce.p12.path > - env.test.gce.project.name > > However the `id` seems to be a password, so it's hidden on the admin > panel. Can you please help me with this? > > [1] > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Tests_Gce&branch_IgniteExtensions_Tests=%3Cdefault%3E&tab=buildTypeStatusDiv > [2] > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_GceOld&branch_IgniteTests24Java8=ignite-2.12&tab=buildTypeStatusDiv > > On Mon, 25 Oct 2021 at 14:22, Maxim Muzafarov wrote: >> >> Folks, >> >> I've moved the azure, gce, aws modules to the ignite-extensions project. >> https://issues.apache.org/jira/browse/IGNITE-15541 >> >> Building the modules in the ignite-extension project will prepare an >> appropriate release zip file containing all the necessary >> dependencies: >> - ignite-aws-ext.zip >> - ignite-gce-ext.zip >> - ignite-auzre-ext.zip >> >> >> On Wed, 13 Oct 2021 at 17:09, Stephen Darlington >> wrote: >>> >>> Okay, I phrased that badly. I mean an extra platform-specific ZIP file that >>> I used to augment the generic Ignite ZIP file. >>> >>> So, to run on Azure I’d download ignite.zip + azure.zip. >>> >>> Extending ignite.sh would also be great, kind of like what’s happening with >>> Ignite 3 as far as I can tell. >>> >>> What I’m advocating is not needing to use Maven just to run Ignite on >>> Azure, AWS, etc. >>> On 13 Oct 2021, at 14:35, Petr Ivanov wrote: Our self-contained zip file currently is over 400Mb and continues to grow. Even considering that internet speeds has grown too, it is nonsense to force user to download such an archive where 90% are useless for most cases. Also we can: — pack all extensions in single binary with latests releases (and update after each extension release) or even one by one — extend ignite.sh to download remote libs when extension is activated via command line Antoine de Saint-Exupéry once said that 'perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away'. We are not obliged to make Apache Ignite ideal, but we certainly can move that way — I am sure the result will exceed expectations. > On 13 Oct 2021, at 16:02, Stephen Darlington > wrote: > > Having extensions in Maven Central makes perfect sense for tools that > need to be built and integrated with other code, Spring integrations for > example. > > That’s not the case for extensions that are required just to run Ignite. > A self-contained zip file for each platform would work. > >> On 13 Oct 2021, at 13:41, Maxim Muzafarov wrote: >> >> Nikolay, >> >> All extensions will be available at the maven central for download. >> >> Previously extensions have a dependent version on the ignite core, so >> each time the Ignite was released it made sense to include all the >> extensions into the uber-zip file. Each extension has its own release >> version now, so an extension can be upgraded and used independently, >> what is the reason include it in the single uber-zip file? Probably it >> would be better to provide a self-contained zip file for each cloud >> platform. >> >> If I've missed your issue, so can you clarify the problem in more detail? >> >> On Wed, 13 Oct 2021 at 14:37, Nikolay Izhikov >> wrote: >>> >>> Maxim. >>> Currently, they are copied from the optional directory of the ignite binary package but would be copied from an appropriate ignite extension binary package. >>> >>> But how, the user will download this binary package? >>> Right now, all the user need is Ignite distributive. >>> >>> 13 окт. 2021 г., в 14:32, Maxim Muzafarov написал(а): Stephen, I guess the required classes of IP-finders should be in the classpath (libs directory). Currently, they are copied from the optional directory of the ignite binary package but would be copied from an appropriate ignite extension binary package. Probably I'm missing something but almost nothing changes in that process from my point of view. The documentation pages will be updated prior to the release. On Wed, 13 Oct 2021 at 13:44, Stephen Darlington >>
Re: Move Azure, AWS, GCE to the ignite-extensions
Petr, There is only the GCE suite left to be configured. I've created an issue [1] to do this. Please, take a look. [1] https://issues.apache.org/jira/browse/IGNITE-15981 On Wed, 24 Nov 2021 at 12:00, Petr Ivanov wrote: > > Hi, Maksim. > > > Can you file a ticket about recreating test suites for extensions, please? > I will attend to it in nearest time. > > > > On 23 Nov 2021, at 17:14, Maxim Muzafarov wrote: > > > > Hello Petr, > > > > Can you assist me with configuring the GCE [1] suite on the TC > > Extensions project? Currently, I have an issue with moving environment > > variables from the old GCE suite [2] to the new one. > > > > I need to create the following envs: > > - env.test.gce.account.id > > - env.test.gce.p12.path > > - env.test.gce.project.name > > > > However the `id` seems to be a password, so it's hidden on the admin > > panel. Can you please help me with this? > > > > [1] > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Tests_Gce&branch_IgniteExtensions_Tests=%3Cdefault%3E&tab=buildTypeStatusDiv > > [2] > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_GceOld&branch_IgniteTests24Java8=ignite-2.12&tab=buildTypeStatusDiv > > > > On Mon, 25 Oct 2021 at 14:22, Maxim Muzafarov wrote: > >> > >> Folks, > >> > >> I've moved the azure, gce, aws modules to the ignite-extensions project. > >> https://issues.apache.org/jira/browse/IGNITE-15541 > >> > >> Building the modules in the ignite-extension project will prepare an > >> appropriate release zip file containing all the necessary > >> dependencies: > >> - ignite-aws-ext.zip > >> - ignite-gce-ext.zip > >> - ignite-auzre-ext.zip > >> > >> > >> On Wed, 13 Oct 2021 at 17:09, Stephen Darlington > >> wrote: > >>> > >>> Okay, I phrased that badly. I mean an extra platform-specific ZIP file > >>> that I used to augment the generic Ignite ZIP file. > >>> > >>> So, to run on Azure I’d download ignite.zip + azure.zip. > >>> > >>> Extending ignite.sh would also be great, kind of like what’s happening > >>> with Ignite 3 as far as I can tell. > >>> > >>> What I’m advocating is not needing to use Maven just to run Ignite on > >>> Azure, AWS, etc. > >>> > On 13 Oct 2021, at 14:35, Petr Ivanov wrote: > > Our self-contained zip file currently is over 400Mb and continues to > grow. > Even considering that internet speeds has grown too, it is nonsense to > force user to download such an archive where 90% are useless for most > cases. > > Also we can: > — pack all extensions in single binary with latests releases (and update > after each extension release) or even one by one > — extend ignite.sh to download remote libs when extension is activated > via command line > > > Antoine de Saint-Exupéry once said that 'perfection is achieved, not > when there is nothing more to add, but when there is nothing left to > take away'. > We are not obliged to make Apache Ignite ideal, but we certainly can > move that way — I am sure the result will exceed expectations. > > > > > On 13 Oct 2021, at 16:02, Stephen Darlington > > wrote: > > > > Having extensions in Maven Central makes perfect sense for tools that > > need to be built and integrated with other code, Spring integrations > > for example. > > > > That’s not the case for extensions that are required just to run > > Ignite. A self-contained zip file for each platform would work. > > > >> On 13 Oct 2021, at 13:41, Maxim Muzafarov wrote: > >> > >> Nikolay, > >> > >> All extensions will be available at the maven central for download. > >> > >> Previously extensions have a dependent version on the ignite core, so > >> each time the Ignite was released it made sense to include all the > >> extensions into the uber-zip file. Each extension has its own release > >> version now, so an extension can be upgraded and used independently, > >> what is the reason include it in the single uber-zip file? Probably it > >> would be better to provide a self-contained zip file for each cloud > >> platform. > >> > >> If I've missed your issue, so can you clarify the problem in more > >> detail? > >> > >> On Wed, 13 Oct 2021 at 14:37, Nikolay Izhikov > >> wrote: > >>> > >>> Maxim. > >>> > Currently, they are copied from the optional > directory of the ignite binary package but would be copied from an > appropriate ignite extension binary package. > >>> > >>> But how, the user will download this binary package? > >>> Right now, all the user need is Ignite distributive. > >>> > >>> > 13 окт. 2021 г., в 14:32, Maxim Muzafarov > написал(а): > > Stephen, > > I guess the required classes of IP-finders should be in the classpa
Moving snapshot tests to their own test suites
Igniters, Currently, there are a lot of snapshot tests running under the `Basic3` test suite. Since the amount of tests is actually growing it is not good to run them at the basic Ignite test suite. I propose moving all the tests to a dedicated test suite and running them independently taking into account that most of them require TEST_SCALE_FACTOR_PROPERTY to be configured. The default number of affinity partitions is 1024 and this is too big for TeamCity agents with slow HDD. I've prepared the PR [2] within the issue [1]. Are there any objections to do such changes? [1] https://issues.apache.org/jira/browse/IGNITE-15233 [2] https://github.com/apache/ignite/pull/9584/files [3] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_DiskPageCompressions1&branch_IgniteTests24Java8=%3Cdefault%3E&tab=buildTypeStatusDiv
IEP-82 Thin Client Retry Policy
Igniters, I've prepared a proposal about thin client retry behavior. Please review and let me know your thoughts. https://cwiki.apache.org/confluence/display/IGNITE/IEP-82+Thin+Client+Retry+Policy
