I've created a JIRA issue for the the Hadoop 2.9.0 build problem [1]. Best, Fabian
[1] https://issues.apache.org/jira/browse/FLINK-8177 2017-11-30 4:35 GMT+01:00 Eron Wright <eronwri...@gmail.com>: > Unfortunately we've identified a blocker bug for Flink on Mesos - > FLINK-8174. We'll have a patch ready on Thursday. > > Thanks, > Eron > > On Wed, Nov 29, 2017 at 3:40 PM, Eron Wright <eronwri...@gmail.com> wrote: > > > On Dell EMC side, we're testing the RC2 on DCOS 1.10.0. Seeing a > > potential issue with offer acceptance and we'll update the thread with a > +1 > > or with a more concrete issue within 24 hours. > > > > Thanks, > > Eron > > > > On Wed, Nov 29, 2017 at 6:54 AM, Chesnay Schepler <ches...@apache.org> > > wrote: > > > >> I don't think anyone has taken a look yet, nor was there a discussion as > >> to postponing it. > >> > >> It just slipped through the cracks i guess... > >> > >> > >> On 29.11.2017 15:47, Gyula Fóra wrote: > >> > >>> Hi guys, > >>> I ran into this again while playing with savepoint/restore parallelism: > >>> > >>> https://issues.apache.org/jira/browse/FLINK-7595 > >>> https://github.com/apache/flink/pull/4651 > >>> > >>> Anyone has some idea about the status of this PR or were we planning to > >>> postpone this to 1.5? > >>> > >>> Thanks, > >>> Gyula > >>> > >>> > >>> Fabian Hueske <fhue...@gmail.com> ezt írta (időpont: 2017. nov. 29., > >>> Sze, > >>> 13:10): > >>> > >>> OK, the situation is the following: > >>>> > >>>> The test class (org.apache.flink.yarn.UtilsTest) implements a Hadoop > >>>> interface (Container) that was extended in Hadoop 2.9.0 by a getter > and > >>>> setter. > >>>> By adding the methods, we can compile Flink for Hadoop 2.9.0. However, > >>>> the > >>>> getter/setter add a dependency on a class that was also added in > Hadoop > >>>> 2.9.0. > >>>> Therefore, the implementation is not backwards compatible with Hadoop > >>>> versions < 2.9.0. > >>>> > >>>> Not sure how we can fix the problem. We would need two version of the > >>>> class > >>>> that are chosen based on the Hadoop version. Do we have something like > >>>> that > >>>> somewhere else? > >>>> > >>>> Since this is only a problem in a test class, Flink 1.4.0 might still > >>>> work > >>>> very well with Hadoop 2.9.0. > >>>> However, this has not been tested AFAIK. > >>>> > >>>> Cheers, Fabian > >>>> > >>>> 2017-11-29 12:47 GMT+01:00 Fabian Hueske <fhue...@gmail.com>: > >>>> > >>>> I just tried to build the release-1.4 branch for Hadoop 2.9.0 > (released > >>>>> a > >>>>> few days ago) and got a compilation failure in a test class. > >>>>> > >>>>> Right now, I'm assessing how much we need to fix to support Hadoop > >>>>> 2.9.0. > >>>>> I'll report later. > >>>>> > >>>>> Best, Fabian > >>>>> > >>>>> 2017-11-29 11:16 GMT+01:00 Aljoscha Krettek <aljos...@apache.org>: > >>>>> > >>>>> Agreed, this is a regression compared to the previous functionality. > I > >>>>>> updated the issue to "Blocker". > >>>>>> > >>>>>> On 29. Nov 2017, at 10:01, Gyula Fóra <gyula.f...@gmail.com> wrote: > >>>>>>> > >>>>>>> Hi all, > >>>>>>> > >>>>>>> I have found the following issue: > >>>>>>> https://issues.apache.org/jira/browse/FLINK-8165 > >>>>>>> > >>>>>>> I would say this is a blocker (I personally pass the ParameterTool > >>>>>>> all > >>>>>>> > >>>>>> over > >>>>>> > >>>>>>> the place in my production apps), but a pretty trivial issue to > fix, > >>>>>>> > >>>>>> we > >>>> > >>>>> can > >>>>>> > >>>>>>> wait a little to find other potential problems. > >>>>>>> > >>>>>>> I can submit a fix in a little bit. > >>>>>>> > >>>>>>> Cheers, > >>>>>>> Gyula > >>>>>>> > >>>>>>> Tzu-Li (Gordon) Tai <tzuli...@apache.org> ezt írta (időpont: 2017. > >>>>>>> > >>>>>> nov. > >>>> > >>>>> 29., Sze, 9:23): > >>>>>>> > >>>>>>> +1 > >>>>>>>> > >>>>>>>> Verified: > >>>>>>>> - No missing release Maven artifacts > >>>>>>>> - Staged Apache source & binary convenience releases looks good > >>>>>>>> - NOTICE / LICENSE is correct, README is sane > >>>>>>>> - Built from source (macOS, Scala 2.11, Hadoop-free & Hadoop 2.8) > >>>>>>>> - Cluster testing on AWS EMR (see release-testing-doc for > >>>>>>>> > >>>>>>> configuration > >>>> > >>>>> details) > >>>>>>>> - Tested Kinesis / Elasticsearch connector (no dependency clashes > on > >>>>>>>> cluster execution, works locally in IDE) > >>>>>>>> > >>>>>>>> Thanks a lot for managing the release Aljoscha! > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> Gordon > >>>>>>>> > >>>>>>>> On 28 November 2017 at 8:32:42 PM, Stefan Richter ( > >>>>>>>> s.rich...@data-artisans.com) wrote: > >>>>>>>> > >>>>>>>> +1 (non-binding) > >>>>>>>> > >>>>>>>> I tested Flink in a cluster setup on Google Cloud, YARN-per-job, > >>>>>>>> > >>>>>>> checked > >>>>>> > >>>>>>> that for all backends that HA, recovery, at-least-once, end-to-end > >>>>>>>> > >>>>>>> exactly > >>>>>> > >>>>>>> once (with Kafka11 Producer), savepoints, externalized checkpoints, > >>>>>>>> > >>>>>>> and > >>>> > >>>>> rescaling work correctly. > >>>>>>>> > >>>>>>>> Am 28.11.2017 um 11:47 schrieb Aljoscha Krettek < > >>>>>>>>> > >>>>>>>> aljos...@apache.org > >>>> > >>>>> : > >>>>>>> > >>>>>>>> +1 > >>>>>>>>> > >>>>>>>>> Verified: > >>>>>>>>> - NOTICE and LICENSE are correct > >>>>>>>>> - source doesn't contain binaries > >>>>>>>>> - verified signatures > >>>>>>>>> - verified hashes > >>>>>>>>> - cluster testing on AWS and Cloudera VM (with Kerberos) (see > >>>>>>>>> > >>>>>>>> release-testing doc) > >>>>>>>> > >>>>>>>>> > >>>>>>>>> On 28. Nov 2017, at 11:20, Aljoscha Krettek <aljos...@apache.org > > > >>>>>>>>>> > >>>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Phew, thanks for the update! > >>>>>>>>>> > >>>>>>>>>> On 28. Nov 2017, at 11:19, Gyula Fóra <gyf...@apache.org> > wrote: > >>>>>>>>>>> > >>>>>>>>>>> Ok seems like I had to remove the snappy jar as it was > corrupted > >>>>>>>>>>> > >>>>>>>>>> (makes > >>>>>> > >>>>>>> total sense) :P > >>>>>>>>>>> > >>>>>>>>>>> Gyula Fóra <gyf...@apache.org> ezt írta (időpont: 2017. nov. > >>>>>>>>>>> 28., > >>>>>>>>>>> > >>>>>>>>>> K, > >>>>>> > >>>>>>> 11:13): > >>>>>>>> > >>>>>>>>> Hi Aljoscha, > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks for the release candidate. I am having a hard time > >>>>>>>>>>>> > >>>>>>>>>>> building > >>>> > >>>>> the rc, > >>>>>>>> > >>>>>>>>> I seem to get this error no matter what I do: > >>>>>>>>>>>> > >>>>>>>>>>>> [ERROR] Failed to execute goal > >>>>>>>>>>>> org.apache.maven.plugins:maven-shade-plugin:2.4.1:shade > >>>>>>>>>>>> > >>>>>>>>>>> (shade-hadoop) on > >>>>>>>> > >>>>>>>>> project flink-shaded-hadoop2-uber: Error creating shaded jar: > >>>>>>>>>>>> > >>>>>>>>>>> invalid > >>>>>> > >>>>>>> LOC > >>>>>>>> > >>>>>>>>> header (bad signature) -> [Help 1] > >>>>>>>>>>>> > >>>>>>>>>>>> (Apache Maven 3.3.9) > >>>>>>>>>>>> > >>>>>>>>>>>> Any idea what I am missing? > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Gyula > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Aljoscha Krettek <aljos...@apache.org> ezt írta (időpont: > 2017. > >>>>>>>>>>>> > >>>>>>>>>>> nov. > >>>>>> > >>>>>>> 27., > >>>>>>>> > >>>>>>>>> H, 19:35): > >>>>>>>>>>>> > >>>>>>>>>>>> Hi everyone, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Please review and vote on release candidate #2 for the > version > >>>>>>>>>>>>> > >>>>>>>>>>>> 1.4.0, as > >>>>>>>> > >>>>>>>>> follows: > >>>>>>>>>>>>> [ ] +1, Approve the release > >>>>>>>>>>>>> [ ] -1, Do not approve the release (please provide specific > >>>>>>>>>>>>> > >>>>>>>>>>>> comments) > >>>>>> > >>>>>>> > >>>>>>>>>>>>> The complete staging area is available for your review, which > >>>>>>>>>>>>> > >>>>>>>>>>>> includes: > >>>>>>>> > >>>>>>>>> * JIRA release notes [1], > >>>>>>>>>>>>> * the official Apache source release and binary convenience > >>>>>>>>>>>>> > >>>>>>>>>>>> releases > >>>>>> > >>>>>>> to > >>>>>>>> > >>>>>>>>> be deployed to dist.apache.org[2], which are signed with the > >>>>>>>>>>>>> > >>>>>>>>>>>> key > >>>> > >>>>> with > >>>>>>>> > >>>>>>>>> fingerprint F2A67A8047499BBB3908D17AA8F4FD97121D7293 [3], > >>>>>>>>>>>>> * all artifacts to be deployed to the Maven Central > Repository > >>>>>>>>>>>>> > >>>>>>>>>>>> [4], > >>>>>> > >>>>>>> * source code tag "release-1.4.0-rc1" [5], > >>>>>>>>>>>>> * website pull request listing the new release [6]. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Please have a careful look at the website PR because I > changed > >>>>>>>>>>>>> > >>>>>>>>>>>> some > >>>>>> > >>>>>>> wording and we're now also releasing a binary without Hadoop > >>>>>>>>>>>>> > >>>>>>>>>>>> dependencies. > >>>>>>>> > >>>>>>>>> Please use this document for coordinating testing efforts: [7] > >>>>>>>>>>>>> > >>>>>>>>>>>>> The only change between RC1 and this RC2 is that the source > >>>>>>>>>>>>> > >>>>>>>>>>>> release > >>>>>> > >>>>>>> package does not include the erroneously included binary Ruby > >>>>>>>>>>>>> > >>>>>>>>>>>> dependencies > >>>>>>>> > >>>>>>>>> of the documentation anymore. Because of this I would like to > >>>>>>>>>>>>> > >>>>>>>>>>>> propose a > >>>>>>>> > >>>>>>>>> shorter voting time and close the vote around the time that RC1 > >>>>>>>>>>>>> > >>>>>>>>>>>> would have > >>>>>>>> > >>>>>>>>> closed. This would mean closing by end of Wednesday. Please let > >>>>>>>>>>>>> > >>>>>>>>>>>> me > >>>> > >>>>> know if > >>>>>>>> > >>>>>>>>> you disagree with this. The vote is adopted by majority > >>>>>>>>>>>>> > >>>>>>>>>>>> approval, > >>>> > >>>>> with at > >>>>>>>> > >>>>>>>>> least 3 PMC affirmative votes. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> Your friendly Release Manager > >>>>>>>>>>>>> > >>>>>>>>>>>>> [1] > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?proje > >>>>>>>> > >>>>>>> ctId=12315522&version=12340533 > >>>>>> > >>>>>>> [2] http://people.apache.org/~aljoscha/flink-1.4.0-rc2/ > >>>>>>>>>>>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > >>>>>>>>>>>>> [4] > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapache > >>>> flink-1140 > >>>> > >>>>> [5] > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h= > >>>>>>>> > >>>>>>> ea751b7b23b23446ed3fcdeed564bbe8bf4adf9c > >>>>>> > >>>>>>> [6] https://github.com/apache/flink-web/pull/95 > >>>>>>>>>>>>> [7] > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://docs.google.com/document/d/1HqYyrNoMSXwo8zBpZj7s39Uz > >>>>>>>> > >>>>>>> UdlFcFO8TRpHNZ_cl44/edit?usp=sharing > >>>>>> > >>>>>>> Pro-tip: you can create a settings.xml file with these contents: > >>>>>>>>>>>>> > >>>>>>>>>>>>> <settings> > >>>>>>>>>>>>> <activeProfiles> > >>>>>>>>>>>>> <activeProfile>flink-1.4.0</activeProfile> > >>>>>>>>>>>>> </activeProfiles> > >>>>>>>>>>>>> <profiles> > >>>>>>>>>>>>> <profile> > >>>>>>>>>>>>> <id>flink-1.4.0</id> > >>>>>>>>>>>>> <repositories> > >>>>>>>>>>>>> <repository> > >>>>>>>>>>>>> <id>flink-1.4.0</id> > >>>>>>>>>>>>> <url> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapache > >>>>>>>> > >>>>>>> flink-1140/ > >>>>>> > >>>>>>> </url> > >>>>>>>>>>>>> </repository> > >>>>>>>>>>>>> <repository> > >>>>>>>>>>>>> <id>archetype</id> > >>>>>>>>>>>>> <url> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapache > >>>>>>>> > >>>>>>> flink-1140/ > >>>>>> > >>>>>>> </url> > >>>>>>>>>>>>> </repository> > >>>>>>>>>>>>> </repositories> > >>>>>>>>>>>>> </profile> > >>>>>>>>>>>>> </profiles> > >>>>>>>>>>>>> </settings> > >>>>>>>>>>>>> > >>>>>>>>>>>>> And reference that in you maven commands via --settings > >>>>>>>>>>>>> path/to/settings.xml. This is useful for creating a > quickstart > >>>>>>>>>>>>> > >>>>>>>>>>>> based > >>>>>> > >>>>>>> on the > >>>>>>>> > >>>>>>>>> staged release and for building against the staged jars. > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>> > >>>>>> > >> > > >
