Just for reference we already run localstack for DynamoDB tests via testcontainers [1] so everything should be ready (at least module wise) to run other simulated ITs. Being able to run all our AWS related ITs with localstack is a really simple and nice contribution to do if anyone is interested.
About real AWS runs there have been conversations about AWS credits donation in the past but sadly with no progress. Most recent ones were at the beginning of this year so it is not a 'forgotten' subject it is just hard to find someone to donate the credits or contribute a credit card for this purpose [2]. Contributions welcome ;) [1] https://www.testcontainers.org/modules/localstack/ [2] https://aws.amazon.com/blogs/opensource/aws-promotional-credits-open-source-projects/ On Tue, Jul 14, 2020 at 11:00 PM Kenneth Knowles <[email protected]> wrote: > > Localstack looks really cool. We have to consider resource consumption and > latency for where it fits in the testing plan. It definitely seems useful for > having ITs easily. For individual IOs, an in-process implementation can be > much more performant without losing much accuracy (since it is also just > another implementation that is not the real one) so if that is available I > would also use one of those (any time you would mock, use an in-process fake > instead). Like direct runner vs ULR. You don't need the ULR to test your > transform's correctness. Localstack's README discusses the pro/con of each > pretty well. Fault injection is pretty compelling and process isolation might > reduce dependency troubles. > > Kenn > > On Fri, Jul 10, 2020 at 4:45 AM Alexey Romanenko <[email protected]> > wrote: >> >> I think that we should get back to this question since that time we have >> more and more AWS-related IO connectors (mostly in Java SDK afaik). >> >> It would be great to have a dedicated Beam’s credentials to run all our >> AWS-releasted ITs against real AWS instance, but till then, I’m +1 to run >> such tests against 3rd party implementations, for example, “localstack”, as >> most comprehensive one. Of course we can observe some potential discrepancy >> in behaviour between real AWS and other implementations, but if it’s not >> principal things then it should not be a stopper. I believe that regular >> running ITs, especially for IO connectors, is a very important. >> >> On 9 Jul 2020, at 22:18, Luke Cwik <[email protected]> wrote: >> >> It has come up a few times[1, 2, 3, 4] and there have also been a few >> comments over time about whether someone could donate AWS resources to the >> project. >> >> 1: https://issues.apache.org/jira/browse/BEAM-601 >> 2: https://issues.apache.org/jira/browse/BEAM-3373 >> 3: https://issues.apache.org/jira/browse/BEAM-3550 >> 4: https://issues.apache.org/jira/browse/BEAM-3032 >> >> On Thu, Jul 9, 2020 at 1:02 PM Mani Kolbe <[email protected]> wrote: >>> >>> Have you guys considered using localstack to run AWS service based >>> integration tests? >>> >>> https://github.com/localstack/localstack >>> >>> On Thu, 9 Jul, 2020, 5:25 PM Piotr Szuberski, <[email protected]> >>> wrote: >>>> >>>> Yeah, I meant KinesisIOIT tests. I'll do the same with the cross-language >>>> it tests then. Thanks for your reply :) >>>> >>>> On 2020/07/08 17:13:11, Alexey Romanenko <[email protected]> wrote: >>>> > If you mean Java KinesisIO tests, then unit tests are running on Jenkins >>>> > [1] and ITs are not running since it requires AWS credentials that we >>>> > don’t have dedicated to Beam for the moment. >>>> > >>>> > In the same time, you can run KinesisIOIT with your own credentials, >>>> > like we do in Talend (a company that I work for). >>>> > >>>> > [1] >>>> > https://ci-beam.apache.org/job/beam_PreCommit_Java_Commit/12209/testReport/org.apache.beam.sdk.io.kinesis/ >>>> > >>>> > <https://ci-beam.apache.org/job/beam_PreCommit_Java_Commit/12209/testReport/org.apache.beam.sdk.io.kinesis/> >>>> > >>>> > > On 8 Jul 2020, at 13:11, Piotr Szuberski <[email protected]> >>>> > > wrote: >>>> > > >>>> > > I'm writing KinesisIO external transform with python wrapper and I >>>> > > found that the tests aren't executed anywhere in Jenkins. Am I wrong >>>> > > or there is a reason for that? >>>> > >>>> > >> >>
