I believe most of this is already handled for various build tools (like the gradle docker plugin) and simplified by docker image repos (bintray, e.g.).
But this is a good point. I'd be interested to see if you've done this before Brett.. On Fri, Jan 12, 2018 at 2:56 PM Gunnar Morling <gun...@hibernate.org> wrote: > Brett, > > What's still unclear to me is when going the Docker route, won't you still > need some code which deploys your tests to Karaf, runs them there and > fetches the test results, so e.g. your Gradle build will fail if there are > test failures? Would you envision to write these bits yourself? And > wouldn't this amount to re-implementing PaxExam yourself? Seems I'm still > missing a piece of the story :) > > Cheers, > > --Gunnar > > > 2018-01-12 19:16 GMT+01:00 Brett Meyer <br...@hibernate.org>: > > > I guess the way I'm looking at this is Docker will be primarily used by > > Jenkins, and myself or anyone working directly on hibernate-osgi > > itself. Otherwise, it'll be disabled by default and hidden behind a > > profile. We'll make sure that most contributors running the entire > > Hibernate test suite won't be affected... > > > > > > On 1/12/18 1:13 PM, andrea boriero wrote: > > > I already have Docker running on my machine, so it seems not a big > > > issue for me,but not sure about the impact for others. > > > > > > Anyway It's worth giving a try. > > > > > > On 12 January 2018 at 17:54, Sanne Grinovero <sa...@hibernate.org > > > <mailto:sa...@hibernate.org>> wrote: > > > > > > On 12 January 2018 at 17:32, Brett Meyer <br...@hibernate.org > > > <mailto:br...@hibernate.org>> wrote: > > > > If I don't have time to contribute to Pax Exam, I certainly > > > don't have > > > > time to start a new project haha... > > > > > > > > And realistically, that "something new" would likely involve > > > containers > > > > anyway. > > > > > > > > At this point, mostly a question of 1) status quo, 2) Docker (or > > any > > > > other container-based solution), or 3) try screwing around with > > > Pax Exam > > > > in "server-only" mode (but I don't have high hopes there). > > > > > > Sure. Docker is now available on the CI slaves too, so that's not > > > a problem. > > > > > > The only annoyance is that the whole ORM team - and anyone > > > contributing - would need to have Docker as well, but that doesn't > > > seem too bad to me... and was likely bound to happen for other > tools > > > :) > > > > > > Steve, Chris and Andrea? Ok with that? Maybe you have Docker > > > running already? > > > > > > > > > > > > > > > On 1/12/18 12:27 PM, Sanne Grinovero wrote: > > > >> Ok, looks like you really should start something new :) > > > >> > > > >> Hopefully many of those other annoyed Karaf developers will > > follow. > > > >> > > > >> On 12 January 2018 at 13:59, Brett Meyer <br...@hibernate.org > > > <mailto:br...@hibernate.org>> wrote: > > > >>> Plus, for me, it's more a question of time. I only have a bit > > > available > > > >>> for open source work these days, and I'd rather spend that > > > knocking out > > > >>> some of the hibernate-osgi tasks we've had on our plate for a > > > while. I > > > >>> unfortunately don't have anything left to contribute to Pax > > > Exam itself, > > > >>> assuming that would even fix the problem. > > > >>> > > > >>> Even worse, we're barely using the integration tests for > > > anything more > > > >>> than a simple smoke test at this point, since it seems like > > > every time > > > >>> we touch it something new goes wrong. Looking for a more > > > *consistent* > > > >>> solution -- need more confidence in the backbone. > > > >>> > > > >>> > > > >>> On 1/12/18 8:56 AM, Brett Meyer wrote: > > > >>>> Sorry Gunnar/Sanne, should have clarified this first: > > > >>>> > > > >>>> We actually used Arquillian before Pax Exam, and the > > > experience was > > > >>>> far worse (somewhat of a long story)... > > > >>>> > > > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, > so I > > > >>>> can't imagine using Karaf without the helpers being a walk in > > > the park > > > >>>> > > > >>>> That's not actually the case. The way Pax Exam currently > > > runs our > > > >>>> tests is fundamentally part of the problem. The test code is > > > >>>> dynamically wrapped in an actual bundle, using something like > > > >>>> tiny-bundles, and executed *within* the container itself. Pax > > > >>>> overrides runs with additional probes, overrides logging > > > >>>> infrastructure, etc. Those nuances can often be the source > > > of many of > > > >>>> the bugs (there are a ton of classloader implications, etc. > > > -- IIRC, > > > >>>> this was one area where Arquillian was much, much worse). > > > There are > > > >>>> some benefits to that setup, but for Hibernate it mainly gets > > > in the way. > > > >>>> > > > >>>> It *does* have a "server mode" where tests run outside of the > > > >>>> container, but I vaguely remember going down that path early > > > on and > > > >>>> hitting a roadblock. For the life of me, I can't remember the > > > >>>> specifics. But my pushback here is that ultimately Docker > > > might be > > > >>>> more preferable, giving us more of a real world scenario to > > > do true > > > >>>> e2e tests without something else in the middle. > > > >>>> > > > >>>>> so I can't imagine using Karaf without the helpers being a > > > walk in > > > >>>> the park; e.g. having to deal with HTTP operations comes with > > > its own > > > >>>> baggage {dependencies, complexity, speed, .. } and generally > > more > > > >>>> stuff to maintain. > > > >>>> > > > >>>> I guess I respectfully disagree with that, but purely due to > > > Karaf > > > >>>> features. Our features.xml does most of the heavy lifting for > > us > > > >>>> w/r/t getting Hibernate provisioned. The same would be true > > > with the > > > >>>> test harness bundle/feature. REST is simple and > > > out-of-the-box thanks > > > >>>> to Karaf + CXF or Camel. For other possible routes (Karaf > > > commands), > > > >>>> we already have code available in our demo/quickstart > projects. > > > >>>> > > > >>>>> Also: considered contributing to Pax? > > > >>>> Yes, of course. But the fact that numerous Karaf *committers* > > > >>>> themselves have a long history of built-up frustration on it > > > doesn't > > > >>>> leave me optimistic. A couple of them had tried to pitch in > > > at one > > > >>>> point and weren't able to get anywhere. > > > >>>> > > > >>>>> but it seems their developers really expect their users to > > > be deeply > > > >>>> familiar with it all > > > >>>> > > > >>>> Absolutely! But again, our struggles also come down to the > > > >>>> fundamental way Pax Exam works... > > > >>>> > > > >>>> > > > >>>> On 1/12/18 6:27 AM, Sanne Grinovero wrote: > > > >>>>> +1 to explore alternatives to Pax Exam, but I'd be wary of > > > maintining > > > >>>>> our own test infrastructure. > > > >>>>> > > > >>>>> Pax Exam was just "helping" to deploy/run things in Karaf, > > > so I can't > > > >>>>> imagine using Karaf without the helpers being a walk in the > > > park; e.g. > > > >>>>> having to deal with HTTP operations comes with its own > baggage > > > >>>>> {dependencies, complexity, speed, .. } and generally more > > > stuff to > > > >>>>> maintain. > > > >>>>> > > > >>>>> So.. +1 to try out Arquillian or anything else. Or maybe you > > > could > > > >>>>> start your own tool, but I'd prefer to see it in a separate > > > repository > > > >>>>> :) e.g. a nice Gradle plugin so maybe you get more people > > > helping? > > > >>>>> > > > >>>>> Also: considered contributing to Pax? My personal experience > > > with it > > > >>>>> has always been a pain but if I had to try identify the > > > reason, it was > > > >>>>> mostly caused by me being unfamiliar with Karaf and not > > > having good > > > >>>>> clues to track down the real failure; maybe some minor error > > > reporting > > > >>>>> improvements could make a big difference to its usability? > Just > > > >>>>> saying, I don't feel like Pax is bad, but it seems their > > > developers > > > >>>>> really expect their users to be deeply familiar with it all > > > - feels > > > >>>>> like the typical case in which they could use some feedback > > > and a > > > >>>>> hand. > > > >>>>> > > > >>>>> Thanks, > > > >>>>> Sanne > > > >>>>> > > > >>>>> On 12 January 2018 at 08:22, Gunnar > > > Morling<gun...@hibernate.org <mailto:gun...@hibernate.org>> wrote: > > > >>>>>> Hi Brett, > > > >>>>>> > > > >>>>>> We also had our fair share of frustration with Pax Exam in > > > HV, and I was > > > >>>>>> (more than once) at the point of dropping it. > > > >>>>>> > > > >>>>>> Docker could work, but as you say it's a bit of a heavy > > > dependency, if not > > > >>>>>> required anyways. Not sure whether I'd like to add this as > > > a prerequisite > > > >>>>>> for the HV build to be executed. And tests in separate > > > profiles tend to be > > > >>>>>> "forgotten" in my experience. > > > >>>>>> > > > >>>>>> One other approach could be to use Arquillian's OSGi > > > support (see > > > >>>>>> https://github.com/arquillian/arquillian-container-osgi > > > <https://github.com/arquillian/arquillian-container-osgi>), did > > > you consider > > > >>>>>> to use that one as an alternative? > > > >>>>>> > > > >>>>>> Cheers, > > > >>>>>> > > > >>>>>> --Gunnar > > > >>>>>> > > > >>>>>> > > > >>>>>> 2018-01-12 3:34 GMT+01:00 Brett Meyer<br...@hibernate.org > > > <mailto:br...@hibernate.org>>: > > > >>>>>> > > > >>>>>>> <tired-rant> > > > >>>>>>> > > > >>>>>>> I'm fed up with Pax Exam and would love to replace it as > the > > > >>>>>>> hibernate-osgi integration test harness. Most of the > > > Karaf committers > > > >>>>>>> I've been working with hate it more than I do. Every > > > single time we > > > >>>>>>> upgrade the Karaf version, something less-than-minor in > > > hibernate-osgi, > > > >>>>>>> upgrade/change dependencies, or attempt to upgrade Pax > > > Exam itself, > > > >>>>>>> there's some new obfuscated failure. And no matter how > > > much I pray, it > > > >>>>>>> refuses to let us get to the container logs to figure out > > what > > > >>>>>>> happened. Tis a house of cards. > > > >>>>>>> > > > >>>>>>> </tired-rant> > > > >>>>>>> > > > >>>>>>> One alternative that recently came up elsewhere: use > > > Docker to bootstrap > > > >>>>>>> the container, hit it with our features.xml, install a > > > test bundle that > > > >>>>>>> exposes functionality externally (over HTTP, Karaf > > > commands, etc), then > > > >>>>>>> hit the endpoints and run assertions. > > > >>>>>>> > > > >>>>>>> Pros: true "integration test", plain vanilla Karaf, direct > > > access to all > > > >>>>>>> logs, easier to eventually support and test other > containers. > > > >>>>>>> > > > >>>>>>> Cons: Need Docker installed for local test runs, probably > > > safer to > > > >>>>>>> isolate the integration test behind a disabled-by-default > > > Maven profile. > > > >>>>>>> > > > >>>>>>> Any gut reactions? > > > >>>>>>> > > > >>>>>>> OSGi is fun and I'm not at all bitter, > > > >>>>>>> > > > >>>>>>> -Brett- > > > >>>>>>> > > > >>>>>>> ;) > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> _______________________________________________ > > > >>>>>>> hibernate-dev mailing list > > > >>>>>>> hibernate-dev@lists.jboss.org > > > <mailto:hibernate-dev@lists.jboss.org> > > > >>>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > >>>>>> _______________________________________________ > > > >>>>>> hibernate-dev mailing list > > > >>>>>> hibernate-dev@lists.jboss.org > > > <mailto:hibernate-dev@lists.jboss.org> > > > >>>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > >>>>> _______________________________________________ > > > >>>>> hibernate-dev mailing list > > > >>>>> hibernate-dev@lists.jboss.org > > > <mailto:hibernate-dev@lists.jboss.org> > > > >>>>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > >>> _______________________________________________ > > > >>> hibernate-dev mailing list > > > >>> hibernate-dev@lists.jboss.org > > > <mailto:hibernate-dev@lists.jboss.org> > > > >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > > > > > > > > > > _______________________________________________ > > > > hibernate-dev mailing list > > > > hibernate-dev@lists.jboss.org <mailto:hibernate-dev@lists. > > jboss.org> > > > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > _______________________________________________ > > > hibernate-dev mailing list > > > hibernate-dev@lists.jboss.org <mailto: > hibernate-dev@lists.jboss.org> > > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > <https://lists.jboss.org/mailman/listinfo/hibernate-dev> > > > > > > > > > > _______________________________________________ > > hibernate-dev mailing list > > hibernate-dev@lists.jboss.org > > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > > _______________________________________________ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
