Hello Nikolay, I've merged the PR to trunk. Thanks for your huge effort and patience going through the review!
Guozhang On Wed, Oct 7, 2020 at 6:52 AM Nikolay Izhikov <nizhi...@apache.org> wrote: > Great news! > Thanks Magnus! > > I’ve updated the PR. > > Looks like we ready to merge it. > > > 7 окт. 2020 г., в 15:29, Magnus Edenhill <mag...@edenhill.se> > написал(а): > > > > Hi, > > > > ducktape v0.8.0 is now released. > > > > Regards, > > Magnus > > > > > > Den ons 7 okt. 2020 kl 10:50 skrev Nikolay Izhikov <nizhi...@apache.org > >: > > > >> Hello. > >> > >> Got 4 approvals for PR [1] > >> The only thing we need to be able to merge it is a ducktape 0.8 release. > >> If ducktape team need any help with the release, please, let me know. > >> > >> [1] https://github.com/apache/kafka/pull/9196 > >> > >> > >>> 21 сент. 2020 г., в 12:58, Nikolay Izhikov <nizhikov....@gmail.com> > >> написал(а): > >>> > >>> Hello. > >>> > >>> I fixed two system tests that fails in trunk, also. > >>> > >>> > streams_upgrade_test.py::StreamsUpgradeTest.test_version_probing_upgrade > >>> streams_static_membership_test.py > >>> > >>> Please, take a look at my PR [1] > >>> > >>> [1] https://github.com/apache/kafka/pull/9312 > >>> > >>>> 20 сент. 2020 г., в 06:11, Guozhang Wang <wangg...@gmail.com> > >> написал(а): > >>>> > >>>> I've triggered a system test on top of your branch. > >>>> > >>>> Maybe you could also re-run the jenkins unit tests since currently all > >> of > >>>> them fails but you've only touched on system tests, so I'd like to > >> confirm > >>>> at least one successful run. > >>>> > >>>> On Wed, Sep 16, 2020 at 3:37 AM Nikolay Izhikov <nizhi...@apache.org> > >> wrote: > >>>> > >>>>> Hello, Guozhang. > >>>>> > >>>>>> I can help run the test suite once your PR is cleanly rebased to > >> verify > >>>>> the whole suite works > >>>>> > >>>>> Thank you for joining to the review. > >>>>> > >>>>> 1. PR rebased on the current trunk. > >>>>> > >>>>> 2. I triggered all tests in my private environment to verify them > after > >>>>> rebase. > >>>>> Will inform you once tests passed on my environment. > >>>>> > >>>>> 3. We need a new ducktape release [1] to be able to merge PR [2]. > >>>>> For now, PR based on the ducktape trunk branch [3], not some > >>>>> specific release. > >>>>> If ducktape team need any help with the release, please, let me > >>>>> know. > >>>>> > >>>>> [1] https://github.com/confluentinc/ducktape/issues/245 > >>>>> [2] https://github.com/apache/kafka/pull/9196 > >>>>> [3] > >>>>> > >> > https://github.com/apache/kafka/pull/9196/files#diff-9235a7bdb1ca9268681c0e56f3f3609bR39 > >>>>> > >>>>>> 16 сент. 2020 г., в 07:32, Guozhang Wang <wangg...@gmail.com> > >>>>> написал(а): > >>>>>> > >>>>>> Hello Nikolay, > >>>>>> > >>>>>> I can help run the test suite once your PR is cleanly rebased to > >> verify > >>>>> the > >>>>>> whole suite works and then I can merge (I'm trusting Ivan and Magnus > >> here > >>>>>> for their reviews :) > >>>>>> > >>>>>> Guozhang > >>>>>> > >>>>>> On Mon, Sep 14, 2020 at 3:56 AM Nikolay Izhikov < > nizhi...@apache.org> > >>>>> wrote: > >>>>>> > >>>>>>> Hello! > >>>>>>> > >>>>>>> I got 2 approvals from Ivan Daschinskiy and Magnus Edenhill. > >>>>>>> Committers, please, join the review. > >>>>>>> > >>>>>>>> 3 сент. 2020 г., в 11:06, Nikolay Izhikov <nizhikov....@gmail.com > > > >>>>>>> написал(а): > >>>>>>>> > >>>>>>>> Hello! > >>>>>>>> > >>>>>>>> Just a friendly reminder. > >>>>>>>> > >>>>>>>> Patch to resolve some kind of technical debt - python2 in system > >> tests > >>>>>>> is ready! > >>>>>>>> Can someone, please, take a look? > >>>>>>>> > >>>>>>>> https://github.com/apache/kafka/pull/9196 > >>>>>>>> > >>>>>>>>> 28 авг. 2020 г., в 11:19, Nikolay Izhikov < > nizhikov....@gmail.com> > >>>>>>> написал(а): > >>>>>>>>> > >>>>>>>>> Hello! > >>>>>>>>> > >>>>>>>>> Any feedback on this? > >>>>>>>>> What I should additionally do to prepare system tests migration? > >>>>>>>>> > >>>>>>>>>> 24 авг. 2020 г., в 11:17, Nikolay Izhikov < > nizhikov....@gmail.com > >>> > >>>>>>> написал(а): > >>>>>>>>>> > >>>>>>>>>> Hello. > >>>>>>>>>> > >>>>>>>>>> PR [1] is ready. > >>>>>>>>>> Please, review. > >>>>>>>>>> > >>>>>>>>>> But, I need help with the two following questions: > >>>>>>>>>> > >>>>>>>>>> 1. We need a new release of ducktape which includes fixes [2], > [3] > >>>>> for > >>>>>>> python3. > >>>>>>>>>> I created the issue in ducktape repo [4]. > >>>>>>>>>> Can someone help me with the release? > >>>>>>>>>> > >>>>>>>>>> 2. I know that some companies run system tests for the trunk on > a > >>>>>>> regular bases. > >>>>>>>>>> Can someone show me some results of these runs? > >>>>>>>>>> So, I can compare failures in my PR and in the trunk. > >>>>>>>>>> > >>>>>>>>>> Results [5] of run all for my PR available in the ticket [6] > >>>>>>>>>> > >>>>>>>>>> ``` > >>>>>>>>>> SESSION REPORT (ALL TESTS) > >>>>>>>>>> ducktape version: 0.8.0 > >>>>>>>>>> session_id: 2020-08-23--002 > >>>>>>>>>> run time: 1010 minutes 46.483 seconds > >>>>>>>>>> tests run: 684 > >>>>>>>>>> passed: 505 > >>>>>>>>>> failed: 9 > >>>>>>>>>> ignored: 170 > >>>>>>>>>> ``` > >>>>>>>>>> > >>>>>>>>>> [1] https://github.com/apache/kafka/pull/9196 > >>>>>>>>>> [2] > >>>>>>> > >>>>> > >> > https://github.com/confluentinc/ducktape/commit/23bd5ab53802e3a1e1da1ddf3630934f33b02305 > >>>>>>>>>> [3] > >>>>>>> > >>>>> > >> > https://github.com/confluentinc/ducktape/commit/bfe53712f83b025832d29a43cde3de3d7803106f > >>>>>>>>>> [4] https://github.com/confluentinc/ducktape/issues/245 > >>>>>>>>>> [5] > >>>>>>> > https://issues.apache.org/jira/secure/attachment/13010366/report.txt > >>>>>>>>>> [6] https://issues.apache.org/jira/browse/KAFKA-10402 > >>>>>>>>>> > >>>>>>>>>>> 14 авг. 2020 г., в 21:26, Ismael Juma <ism...@juma.me.uk> > >>>>> написал(а): > >>>>>>>>>>> > >>>>>>>>>>> +1 > >>>>>>>>>>> > >>>>>>>>>>> On Fri, Aug 14, 2020 at 7:42 AM John Roesler < > >> vvcep...@apache.org> > >>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Thanks Nikolay, > >>>>>>>>>>>> > >>>>>>>>>>>> No objection. This would be very nice to have. > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> John > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Aug 14, 2020, at 09:18, Nikolay Izhikov wrote: > >>>>>>>>>>>>> Hello. > >>>>>>>>>>>>> > >>>>>>>>>>>>>> If anyone's interested in porting it to Python 3 it would > be a > >>>>> good > >>>>>>>>>>>> change. > >>>>>>>>>>>>> > >>>>>>>>>>>>> I’ve created a ticket [1] to upgrade system tests to python3. > >>>>>>>>>>>>> Does someone have any additional inputs or objections for > this > >>>>>>> change? > >>>>>>>>>>>>> > >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-10402 > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>>> 1 июля 2020 г., в 00:26, Gokul Ramanan Subramanian < > >>>>>>>>>>>> gokul24...@gmail.com> написал(а): > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks Colin. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> While at the subject of system tests, there are a few times > I > >> see > >>>>>>> tests > >>>>>>>>>>>>>> timed out (even on a large machine such as m5.4xlarge EC2 > with > >>>>>>> Linux). > >>>>>>>>>>>> Are > >>>>>>>>>>>>>> there any knobs that system tests provide to control > timeouts > >> / > >>>>>>>>>>>> throughputs > >>>>>>>>>>>>>> across all tests? > >>>>>>>>>>>>>> Thanks. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Tue, Jun 30, 2020 at 6:32 PM Colin McCabe < > >> cmcc...@apache.org > >>>>>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Ducktape runs on Python 2. You can't use it with Python 3, > >> as > >>>>>>> you are > >>>>>>>>>>>>>>> trying to do here. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> If anyone's interested in porting it to Python 3 it would > be > >> a > >>>>>>> good > >>>>>>>>>>>> change. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Otherwise, using docker as suggested here seems to be the > >> best > >>>>>>> way to > >>>>>>>>>>>> go. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> best, > >>>>>>>>>>>>>>> Colin > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Mon, Jun 29, 2020, at 02:14, Gokul Ramanan Subramanian > >> wrote: > >>>>>>>>>>>>>>>> Hi. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Has anyone had luck running Kafka system tests on a Mac. I > >>>>> have a > >>>>>>>>>>>> MacOS > >>>>>>>>>>>>>>>> Mojave 10.14.6. I got Python 3.6.9 using pyenv. However, > the > >>>>>>> command > >>>>>>>>>>>>>>>> *ducktape tests/kafkatest/tests* yields the following > error, > >>>>>>> making > >>>>>>>>>>>> it > >>>>>>>>>>>>>>> look > >>>>>>>>>>>>>>>> like some Python incompatibility issue. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> $ ducktape tests/kafkatest/tests > >>>>>>>>>>>>>>>> Traceback (most recent call last): > >>>>>>>>>>>>>>>> File "/Users/gokusubr/.pyenv/versions/3.6.9/bin/ducktape", > >> line > >>>>>>> 11, > >>>>>>>>>>>> in > >>>>>>>>>>>>>>>> <module> > >>>>>>>>>>>>>>>> load_entry_point('ducktape', 'console_scripts', > >> 'ducktape')() > >>>>>>>>>>>>>>>> File > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>> > >>>>> > >> > "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", > >>>>>>>>>>>>>>>> line 487, in load_entry_point > >>>>>>>>>>>>>>>> return get_distribution(dist).load_entry_point(group, > name) > >>>>>>>>>>>>>>>> File > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>> > >>>>> > >> > "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", > >>>>>>>>>>>>>>>> line 2728, in load_entry_point > >>>>>>>>>>>>>>>> return ep.load() > >>>>>>>>>>>>>>>> File > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>> > >>>>> > >> > "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", > >>>>>>>>>>>>>>>> line 2346, in load > >>>>>>>>>>>>>>>> return self.resolve() > >>>>>>>>>>>>>>>> File > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>> > >>>>> > >> > "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", > >>>>>>>>>>>>>>>> line 2352, in resolve > >>>>>>>>>>>>>>>> module = __import__(self.module_name, > fromlist=['__name__'], > >>>>>>>>>>>>>>>> level=0) > >>>>>>>>>>>>>>>> File > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>> > >>>>> > >> > "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/ducktape-0.7.6-py3.6.egg/ducktape/command_line/main.py", > >>>>>>>>>>>>>>>> line 127 > >>>>>>>>>>>>>>>> print "parameters are not valid json: " + str(e.message) > >>>>>>>>>>>>>>>> ^ > >>>>>>>>>>>>>>>> SyntaxError: invalid syntax > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> I followed the instructions in tests/README.md to setup a > >>>>>>> cluster of > >>>>>>>>>>>> 9 > >>>>>>>>>>>>>>>> worker machines. That worked well. When I ran *python > >> setup.py > >>>>>>>>>>>> develop* > >>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>> install the necessary dependencies (including ducktape), I > >> got > >>>>>>>>>>>> similar > >>>>>>>>>>>>>>>> errors to above, but the overall command completed > >>>>> successfully. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Any help appreciated. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> -- > >>>>>> -- Guozhang > >>>>> > >>>>> > >>>> > >>>> -- > >>>> -- Guozhang > >>> > >> > >> > > -- -- Guozhang
