+d...@kafka.apache.org On Wed, Dec 26, 2018 at 8:53 PM Parviz deyhim <dey...@gmail.com> wrote:
> Thanks fair points. Probably best if I simplify the question: How does > Kafka community run tests besides using mocked local Kafka components? > Surely there are tests to confirm different failure scenarios such as > losing a broker in a real clustered environment (multi node cluster with > Ip, port, hostnsmes and etc). The answer would be a good starting point for > me. > > On Wed, Dec 26, 2018 at 6:11 PM Stephen Powis <spo...@salesforce.com> > wrote: > >> Without looking into how the integration tests work my best guess is >> within >> the context they were written to run in, it doesn't make sense to run them >> against a remote cluster. The "internal" cluster is running the same >> code, >> so why require having to coordinate with an external dependency? >> >> For the use case you gave, and I'm not sure if tests exist that cover this >> behavior or not -- Running the brokers locally in the context of the tests >> mean that those tests have control over the brokers (IE shut them off, >> restart them, etc.. programmatically) and validate behavior. To >> coordinate >> these operations on a remote broker would be significantly more difficult. >> >> Not sure this helps...but perhaps you're either asking the wrong questions >> or trying to go about solving your problem using the wrong set of tools? >> My gut feeling says if you want to do a full scale multi-server load / HA >> test, Kafka's test suite is not the best place to start. >> >> Stephen >> >> >> >> On Thu, Dec 27, 2018 at 10:53 AM Parviz deyhim <dey...@gmail.com> wrote: >> >> > Hi, >> > >> > I'm looking to see who has done this before and get some guidance. On >> > frequent basis I like to run basic tests on a remote Kafka cluster while >> > some random chaos/faults are being performed. In other words I like to >> run >> > chaos engineering tasks (network outage, disk outage, etc) and see how >> > Kafka behaves. For example: >> > >> > 1) bring some random Broker node down >> > 2) send 2000 messages >> > 3) consumes messages >> > 4) confirm there's no data loss >> > >> > My questions: I'm pretty sure most of the scenarios I'm looking to test >> > have been covered under Kafka's integration,unit and other existing >> tests. >> > What I cannot figure out is how to run those tests on a remote cluster >> vs. >> > a local one which the tests seems to run on. For example I like to run >> the >> > following command but the tests to be executed on a remote cluster of my >> > choice: >> > >> > ./gradlew cleanTest integrationTest >> > >> > Any guidance/help would be appreciated. >> > >> > Thanks >> > >> >
