I hadn't seen Testcontainers before. That looks pretty cool and I think it would be a great idea to explore. It seems like having a stable test infrastructure in Docker like Francis has been working on is a good first step.
-- Michael Mior mm...@uwaterloo.ca Le mar. 15 mai 2018 à 17:26, Kevin Risden <kris...@apache.org> a écrit : > One idea I've been throwing around in my head is > https://www.testcontainers.org/ where it would let you run Docker > containers within unit tests. Again only an idea at this point but could > move the actual integration tests back into Calcite instead of it being > completely separate? > > Kevin Risden > > On Thu, May 3, 2018 at 1:32 AM, Francis Chuang <francischu...@apache.org> > wrote: > > > Hey Christina, > > > > Thanks for clarifying! I think it's possible to ask infrastructure set up > > a docker repository for us, however, it also means that it's something we > > need to maintain vs using the official geode images. I am not sure what > the > > policy is around releasing an artifact and docker image for testing that > is > > not part of an "official release", though. > > > > Ideally, it would be better if GEODE-3971 is fixed to avoid this problem. > > How long does the spring app take to download dependencies and compile? > If > > it's not too long, perhaps we can use the maven docker image and compile > > the spring app when we run the integration tests. > > > > Francis > > > > > > On 3/05/2018 3:37 PM, Christian Tzolov wrote: > > > >> Hi Francis, > >> > >> Regarding Geode, initially i have tried to to ingest the test data via > the > >> Geode's REST/JSON endpoint but bumped into this bug: > >> https://issues.apache.org/jira/browse/GEODE-3971 (still unresolved). > >> > >> As consequence i had to write a custom ingestion using Geode Java API. > >> But > >> since i had to compile and run this tool, it might no much sense to > deploy > >> and maintain the geode cluster via scripts (or docker image). Instead > it > >> is simpler to embed the entire Geode cluster into the same (standalone) > >> application. Because this is a Spring Boot app if we deploy the > pre-build > >> jar to a public maven repo we can easy create a Docker image that runs > it > >> in one line (e.g. java -jar ./....) > >> > >> Any ideas where we can host this project and where to release it? > >> > >> Cheers, > >> Christian > >> > >> > >> On 23 April 2018 at 14:12, Francis Chuang <francischu...@apache.org> > >> wrote: > >> > >> Thanks, Michael! > >>> > >>> I noticed that I forgot the link to my fork in my original message. > Here > >>> is my fork if someone wants to hack on it a bit more: > >>> > https://github.com/Boostport/calcite-test-dataset/tree/switch-to-docker > >>> > >>> > >>> On 23/04/2018 9:58 PM, Michael Mior wrote: > >>> > >>> Thanks for raising this Francis. I was hoping to find more time to > spend > >>>> on > >>>> this but unfortunately that hasn't happened. > >>>> > >>>> 1. That's a question for Christian Tzolov. I'm not too familiar with > >>>> Geode. > >>>> 2. You are correct that the VM contains several different database > >>>> servers > >>>> with various ports exposed. I'm not sure what the situation is with > >>>> HSQLDB/H2. > >>>> 3. Maven is definitely not strictly necessary although some of the > >>>> dependencies currently pull in datasets that are used for some of the > >>>> DBs > >>>> before building the VM. > >>>> 4. I don't really have a strong preference either way. I'm sure > someone > >>>> else can speak to why this was separated in the first place. > >>>> > >>>> > >>>> -- > >>>> Michael Mior > >>>> mm...@uwaterloo.ca > >>>> > >>>> 2018-04-23 7:11 GMT-04:00 Francis Chuang <francischu...@apache.org>: > >>>> > >>>> There is currently an issue open for this in the calcite-test-dataset > >>>> > >>>>> repository[1], however, I would like to hear more from the wider > >>>>> community > >>>>> regarding this. > >>>>> > >>>>> I have created a `switch-to-docker` branch on my fork and committed a > >>>>> docker-compose.yml under the docker folder, but ran into a few > >>>>> roadblocks > >>>>> and didn't have any more time to investigate. > >>>>> > >>>>> I am currently investigating using docker-composer to orchestrate and > >>>>> set > >>>>> up the containers. > >>>>> > >>>>> Questions: > >>>>> > >>>>> 1. I am not very familiar with Apache Geode. I was able to start the > >>>>> server and locator using the official docker image, but there does > not > >>>>> appear to be anyway to import data. In the current repository, > there's > >>>>> some > >>>>> java code in `geode-standalone-cluster`. Why do/did we need to write > >>>>> Java > >>>>> code to stand up a geode cluster? Does anyone know if there are any > >>>>> standalone tools (preferably something with built binaries) that we > can > >>>>> use > >>>>> to directly ingest the JSON data? > >>>>> > >>>>> 2. From my reading of the integration test instructions[2], the > >>>>> calcite-test-dataset spins up a VM with databases preloaded with data > >>>>> which > >>>>> the main calcite repository runs tests against. HSQLDB and H2 does > not > >>>>> have > >>>>> any open ports in the VM that's spun up. How does does Calcite run > >>>>> tests > >>>>> against HSQLDB and H2? > >>>>> > >>>>> 3. What is the role of maven in the calcite-test-dataset repository? > I > >>>>> see > >>>>> a lot of POMs in various subfolders such as mysql, postgresql, etc. > >>>>> However, I am not sure what these do. If maven is used to spin up the > >>>>> VM, > >>>>> perhaps we could remove the dependency on it and just run a > >>>>> `docker-compose > >>>>> up` to start the network of containers. > >>>>> > >>>>> 4. Is there any interest in bringing the contents of > >>>>> calcite-test-dataset > >>>>> directly into the Calcite repo? The repo zips up to 1.5MB, so it > might > >>>>> not > >>>>> bring to much bloat to the Calcite repo. > >>>>> > >>>>> Francis > >>>>> > >>>>> [1] https://github.com/vlsi/calcite-test-dataset/issues/8 > >>>>> > >>>>> [2] https://calcite.apache.org/docs/howto.html#running-integrati > >>>>> on-tests > >>>>> > >>>>> > >>>>> > >>>>> > >> > > >
