Hi Igor, > Also, I observed moving calcite integration tests to an in-memory database(MongoDB, ElasticSearch, Cassandra). Why don't I like it? I would like to address some concerns you have raised with recent changes to adapter testing in calcite.
The main drive for those changes was to run adapter tests during merge (PR) phase and fail early if there is any problem. Adapter tests (elastic / mongo / cassandra) are now run as part of CI. Before, to tests adapters, one had to manually run integration tests (which was often forgotten or not run for all adapters). So bugs were caught later (before release?). For the purpose of this discussion let's divide "test" databases into fakes (eg. fongo, which is independent implementation) and embedded (eg. elastic, which uses identical software). And, I believe, you dislike fakes and OK with embedded ? In my opinion fakes are still useful and important for the following reasons: 1) Tests can run from any IDE / maven without container dependency (Vagrant / VirtualBox / docker etc.). For some of us (working in corporate environment) installing docker is not an option. 2) No dependency on external resource (database). This means you don't need to manually (or programmatically) start / stop processes. 3) Tests run as part of CI build. That being said I'm not suggesting to drop integration tests in favor of fakes, on the contrary. I believe it is important to test your software against "real" database (in addition to mocks and fakes). You will notice that Mongo has two tests: MongoAdapterIT <https://github.com/apache/calcite/blob/master/mongodb/src/test/java/org/apache/calcite/test/MongoAdapterIT.java> (which runs against real Mongo during integration tests) and MongoAdapterTest <https://github.com/apache/calcite/blob/master/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java> (which runs fongo during unit tests). They _mostly_ share same tests. It is also true that fongo doesn't support all Mongo functionalities for this cases we mark the test as "run on real mongo only" example testCountGroupByEmptyMultiplyBy2 <https://github.com/apache/calcite/blob/master/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoAdapterTest.java#L392> (which is run when connected to "real" mongo). Regards, Andrei. On Mon, Aug 13, 2018 at 8:11 AM Francis Chuang <[email protected]> wrote: > Thank you so much for working on this, Igor! > > Migrating from Vagrant to docker was also something I looked at at the > beginning of the year, but I hit a few roadblocks and it sort of fell of > my radar. > > It's getting late so, so I won't have a chance to try it out tonight, > but looking at the commits, it does look good! > > I am wondering if it would be possible to move all of this directly into > the Calcite repo, as it would make testing much easier. For example, > after starting a vote, it'd be possible to download the tar, untar it > and run a single command inside the directory to automatically run all > tests + integration tests. > > Francis > > On 13/08/2018 8:46 PM, Igor Kryvenko wrote: > > Hi all, last few months I worked on moving current test environment to > > docker environment. > > Thanks, Volodymyr Vysotskiy for the initial patch. > > > > *Motivation* > > > > I noticed that the current test environment has problems with updating > > versions of databases and often OOM. > > I investigated previous tickets about moving to Docker environment, and > > there was only one problem that there was no stable docker for Mac OS and > > Windows. > > Now, As far as I know, it works stable for them, and we can use it. > > > > Also, I observed moving calcite integration tests to an in-memory > > database(MongoDB, ElasticSearch, Cassandra). Why don't I like it? > > In case of MongoDB, we use Fongo library, which has no full support of > all > > features of MongoDB, so it creates one more dependency for calcite. > > Before, we need just implement some feature in calcite and use latest > > MongoDB with this feature. Now we use Fongo, and if we want to support > the > > latest features of MongoDB, Fongo has to implement them also. > > In the case of ElasticSearch, I think it is the comfortable tradeoff > > because we use official ElasticSearch API to construct an in-memory > > database. > > > > Also, there is one more advantage of using Docker, that if we just make > > changes in some module(e.g. Cassandra) we can just start docker only with > > Cassandra image, we don't need to setup whole virtual machine with all > > databases. > > Also setting up all docker images is faster that Vagrant, even if we > launch > > it the first time. > > Next launching will be very fast, thanks to Docker's cache until we > change > > the context of the Docker container. > > > > > > I've already created the branch in calcite-test-dataset -- > > https://github.com/igorKryvenko/calcite-test-dataset/tree/docker-new > > <https://github.com/igorKryvenko/calcite-test-dataset/tree/docker-new> > > and branch in calcite-project with corresponding changes(a few changes, > but > > I need someone's look at it) --- > > https://github.com/igorKryvenko/calcite/tree/docker > > > > > > I will be appreciated if someone will test my changes on Windows and Mac > > OS. > > > > *Please, do not hesitate and post your questions and remarks.* > > > > Kind regards > > Igor Kryvenko > > > >
