It seems that gitbox.apache.org is not available on CI/CD.
I will try to update VCS to GitHub.
> On 28 Sep 2020, at 14:07, Alex Plehanov <plehanov.a...@gmail.com> wrote:
>
> Guys,
>
> I've tried to build release candidate using TC [1]. But:
> 1. I can't choose a branch in this TC task (there is no such field).
> 2. On the "default" branch I got an error: Failed to collect changes,
> error: List remote refs failed:
> org.apache.http.conn.ConnectTimeoutException: Connect to
> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed: connect
> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> internal id=74, parent id=GitBoxAsfIgnite, description: "
> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>
> Is there some problem with this TC task? What am I doing wrong?
>
> [1] :
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>
> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <alexey.goncha...@gmail.com>:
>
>> Saikat, Nikolay,
>>
>> We have migrated a bunch of modules to ignite-extensions recently. Given
>> that these modules will not be available in Ignite 2.9 anymore (will
>> they?), should we also plan to release the extensions?
>>
>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <plehanov.a...@gmail.com>:
>>
>>> Igniters,
>>>
>>> I've prepared release notes for the 2.9 release [1]. Can anyone review
>> it,
>>> please?
>>>
>>> [1]: https://github.com/apache/ignite/pull/8273
>>>
>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <plehanov.a...@gmail.com>:
>>>
>>>> Guys,
>>>>
>>>> I've filled the ticket with reproducer [1] for the discovery bug. This
>>> bug
>>>> caused by [2] ticket. We discussed with Vladimir Steshin privately and
>>>> decided to revert this ticket. I will do it today (after TC bot visa)
>> if
>>>> there are no objections.
>>>>
>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>>>
>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <plehanov.a...@gmail.com>:
>>>>
>>>>> Guys,
>>>>>
>>>>> During internal testing, we've found a critical bug with
>>>>> discovery (cluster falls apart if two nodes segmented sequentially).
>>> This
>>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>>>> before release. Under investigation now. I'll let you know when we get
>>>>> something.
>>>>>
>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>>>>
>>>>>>> So what do you think? Should we proceed with a 'hacked' version of
>>> the
>>>>>> message factory in 2.9 and go for the runtime message generation in
>>> later
>>>>>> release, or keep the code clean and fix the regression in the next
>>> releases?
>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>> straightforward and does not change the semantics, just skips the
>>> factory
>>>>>> closures for certain messages.
>>>>>>
>>>>>> IMHO 2.5% isn't too much especially because it isn't actual for all
>>>>>> workloads (I didn't get any significant drops during benchmarking).
>> So
>>>>>> I prefer the runtime generation in later releases.
>>>>>>
>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>>>>> <alexey.goncha...@gmail.com> wrote:
>>>>>>>
>>>>>>> Alexey, Andrey, Igniters,
>>>>>>>
>>>>>>> So what do you think? Should we proceed with a 'hacked' version of
>>> the
>>>>>> message factory in 2.9 and go for the runtime message generation in
>>> later
>>>>>> release, or keep the code clean and fix the regression in the next
>>> releases?
>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>> straightforward and does not change the semantics, just skips the
>>> factory
>>>>>> closures for certain messages.
>>>>>>>
>>>>>>> Personally, I would prefer fixing the regression given that we also
>>>>>> introduced tracing in this release.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>> plehanov.a...@gmail.com
>>>> :
>>>>>>>>
>>>>>>>> Alexey,
>>>>>>>>
>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95
>>> and
>>>>>> the performance of ed52559eb95 is better for about 2.5% on average on
>>> our
>>>>>> environment (about the same results we got benchmarking 65c30ec6 vs
>>>>>> 0606f03d). We've made 24 runs for each commit of
>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
>>> clients
>>>>>> 50 threads each. Yardstick results for this configuration:
>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>>>>>>>
>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>>>>> a.budnikov.ign...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>> I posted an instruction on how to publish the docs on
>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
>>>>>> update the docs by following the instruction. Unfortunately, I won't
>> be
>>>>>> able to spend any time on this project any longer. You can send your
>>> pull
>>>>>> requests and questions about the documentation to Denis Magda.
>>>>>>>>>
>>>>>>>>> -Artem
>>>>>>>>>
>>>>>>>>> [1] :
>>>>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>>>>>>>
>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>>>>> alexey.goncha...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Alexey,
>>>>>>>>>>
>>>>>>>>>> I've tried to play with message factories locally, but
>>>>>> unfortunately, I
>>>>>>>>>> cannot spot the difference between old and new implementation in
>>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>>>>> MessageFactoryImpl
>>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>>> branch
>>>>>>>>>> (discussed this with Andrey Gura, the change should be
>> compatible
>>>>>> with the
>>>>>>>>>> new metrics as we still use the register() mechanics).
>>>>>>>>>>
>>>>>>>>>> Can you check if this change makes any difference
>> performance-wise
>>>>>> in your
>>>>>>>>>> environment? If yes, we can go with runtime code generation in
>> the
>>>>>> long
>>>>>>>>>> term: register classes and generate a dynamic message factory
>> with
>>>>>> a switch
>>>>>>>>>> statement once all messages are registered (not in 2.9 though,
>>>>>> obviously).
>>>>>>>>>>
>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>> plehanov.a...@gmail.com
>>>>>>> :
>>>>>>>>>>
>>>>>>>>>>> Hello guys,
>>>>>>>>>>>
>>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]), it
>>>>>> reduced the
>>>>>>>>>>> drop, but not completely removed it.
>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the patch?
>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2] against
>>>>>> 2.8.1
>>>>>>>>>>> release on your environment?
>>>>>>>>>>> With this patch on our environment, it's about a 3% drop left,
>>>>>> it's close
>>>>>>>>>>> to measurement error and I think such a drop is not a
>>>>>> showstopper. Guys,
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>> driver
>>>>>> between 2.8
>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>> should
>>>>>> be
>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>>>>>>>>>>
>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>>> message
>>>>>>>>>>> send problem in some circumstances). I will cherry-pick it if
>>>>>> there is no
>>>>>>>>>>> objection.
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>>>>>>>>>
>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>> mmu...@apache.org
>>>> :
>>>>>>>>>>>
>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>
>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>> this
>>>>>> issue is
>>>>>>>>>>>> related to the new master key change functionality which
>>>>>> haven't been
>>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>> to
>>>>>> the
>>>>>>>>>>>> release branch.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>>>>> nizhi...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello, Igniters.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>> [1]
>>>>>> into the
>>>>>>>>>>> 2.9
>>>>>>>>>>>> release
>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
>>> in
>>>>>> wrong
>>>>>>>>>>>> order since the 2 row when fields_count>10"
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>>>>>>>>>>> [2]
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>>>>>>>>>>>
>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>>>>>>>>> alexey.goncha...@gmail.com>
>>>>>>>>>>>> написал(а):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>>>>> anything out of
>>>>>>>>>>>> the
>>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>>>>>> right now),
>>>>>>>>>>> so
>>>>>>>>>>>>>> most likely the only move here is to switch back to the
>>>>>> switch
>>>>>>>>>>> approach
>>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>>> the
>>>>>> Ignite
>>>>>>>>>>>> messages
>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>>> me
>>>>>> explore
>>>>>>>>>>> the
>>>>>>>>>>>>>> code a bit.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>> the
>>>>>>>>>>> performance.
>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>>>>>> virtual call
>>>>>>>>>>>>>> should not make that much of a difference given the
>> amount
>>>>>> of other
>>>>>>>>>>>> work
>>>>>>>>>>>>>> happening during the message processing.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>>>>> plehanov.a...@gmail.com
>>>>>>>>>>>> :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>> results.
>>>>>>>>>>> Actually,
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>> between
>>>>>> e6a7f93
>>>>>>>>>>>> (first
>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>>>>> 6592dfa5 (last
>>>>>>>>>>>> commit in
>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>>>>> commits.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>> a
>>>>>> drop
>>>>>>>>>>> between
>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>>>>> reverted
>>>>>>>>>>>> IGNITE-13060
>>>>>>>>>>>>>>> now and performance looks the same)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>>>>> related to
>>>>>>>>>>>> drop
>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>>>>> problem, and
>>>>>>>>>>> we
>>>>>>>>>>>> can
>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>> leave
>>>>>> it as is?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>>>>> refactoring)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>>>>>>>>>> alexey.goncha...@gmail.com
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>>>>>> incorrect fix
>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>>>>>> [1], so it
>>>>>>>>>>>> cannot be
>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>> work
>>>>>> with fix
>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>>>>> versions.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --AG
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>>>>>>>>>> alexey.goncha...@gmail.com
>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>>>>>>>>> plehanov.a...@gmail.com
>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>>>>> IGNITE-12568
>>>>>>>>>>>> (reverted
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>>>>> paths, to
>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>> here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>> switch/case
>>>>>> block was
>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>>>>> message factory
>>>>>>>>>>>> is on
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>> impact
>>>>>> on total
>>>>>>>>>>>>>>>>>> performance.
>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>>>>> microbenchmarks,
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> found
>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>>>>> about 30-35%
>>>>>>>>>>>> faster
>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>>> can
>>>>>>>>>>> effectively
>>>>>>>>>>>>>>>>>> inline
>>>>>>>>>>>>>>>>>> message creation code, but with an array of
>> suppliers
>>>>>> relatively
>>>>>>>>>>>> heavy
>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>>>>> rewrite the
>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>> interface
>>>>>> (to
>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>>>>>> but it gives
>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>> looks
>>>>>> ugly
>>>>>>>>>>>> (lambdas
>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>>>>> optimize the
>>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>>>>> Andrey Gura,
>>>>>>>>>>> as
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>> about
>>>>>>>>>>>> optimization?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>>>>>> some metrics
>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>>>>>> factory
>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>>>>> already released
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>>>>> only present
>>>>>>>>>>>> in Ignite
>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>>>>>> metrics
>>>>>>>>>>>> created for
>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>>>>>> the release
>>>>>>>>>>>> and deal
>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>>>
>>>
>>
