Hi everyone, I read the doc, and found that we are not even testing Flink against Kinesis with *flink-connector-kinesis*.
According to our experience, *flink-connector-kinesis *releases tend to have more issues w.r.t. dependency version conflicts, performance, capability, and stability. We've reported 5 issues (FLINK-6365, 6951, 7223, 7366, 7367) so far and resolved one (FLINK-6365). I guess those issues are resulting from 1) flink-connector-kinesis is not built as part of Flink's building process (because of AWS licensing issue), and 2) release managers are not testing flink-connector-kinesis in release testing pipeline. Thus, we cannot detect those issues before releasing. @StephanEwen and I talked about the popularity of Kinesis within Flink users, Stephan said Kinesis is quite *popular*. If it's a popular one, shall we include it in the testing plan from now on? Thanks, Bowen On Fri, Aug 4, 2017 at 8:36 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > +1 > > - checked checksums and signatures > - did extensive cluster testing on GCE with dataproc and a lot of > different combinations of backends and configuration settings > > Tested configuration combinations (from the testing coordination document): > - (aljoscha, GCE, dataproc, Rocks(incremental), YARN per-job (no HA), no > kerberos, Kafka 0.10.x, OK) > - (aljoscha, GCE, dataproc, Rocks(no incremental), YARN per-job (no HA), > no kerberos, Kafka 0.10.x, OK) > - (aljoscha, GCE, dataproc, Rocks(incremental), YARN per-job (HA), no > kerberos, Kafka 0.10.x, OK) > - (aljoscha, GCE, dataproc, Rocks(no incremental), YARN per-job (HA), no > kerberos, Kafka 0.10.x, OK) > - (aljoscha, GCE, dataproc, FileBackend(sync), YARN per-job (no HA), no > kerberos, Kafka 0.10.x, ONGOING) > - (aljoscha, GCE, dataproc, FileBackend(async), YARN per-job (HA), no > kerberos, Kafka 0.10.x, ONGOING) > > > On 4. Aug 2017, at 16:57, Fabian Hueske <fhue...@gmail.com> wrote: > > > > +1 > > > > - checked added and modified Maven dependencies since Flink 1.3.1 > > - io.dropwizard.metrics/metrics-core was upgraded to 3.1.5 and is ASL > > - org.xerial.snappy/snappy-java/1.0.5 was added and is ASL > > - checked flink-runtime-web for new files with non-Apache licenses > > > > Cheers, Fabian > > > > 2017-08-04 13:32 GMT+02:00 Tzu-Li (Gordon) Tai <tzuli...@apache.org>: > > > >> +1 > >> > >> - Functional cluster test results carried over from RC2 (did not retest > >> those for RC3 as the additional commit fix for Gelly should be > irrelevant) > >> - Built from source with Scala 2.10 and 2.11 (MacOSX), successful > >> - Checked to-be-released Maven artifacts > >> > >> Cheers, > >> Gordon > >> > >> On 4 August 2017 at 5:43:09 PM, Till Rohrmann (trohrm...@apache.org) > >> wrote: > >> > >> +1 > >> > >> - Checked checksums and GPG files > >> - Sources contain no binaries > >> - Check all POM files point to the same version > >> - Read readme file > >> - Build from source archive with Hadoop 2.8.1 > >> - Test start up scripts for local and standalone mode > >> - Test job submission for multi-node cluster > >> - Plan-visualizer > >> - Verified quickstarts for IntelliJ and Eclipse > >> - Tested SBT > >> > >> Caveat but not a blocker: > >> - Building with Maven 3.3+ with Hadoop 2.7+ fails because Hadoop 2.7+ > pulls > >> in Curator 2.7.1 which makes the ZooKeeperTests fail with > >> NoSuchMethodError. The problem are the immutable reactor builds. > Installing > >> first the Hadoop-shaded dependencies and then running the build again > >> solves the problem. > >> > >> Cheers, > >> Till > >> > >> On Fri, Aug 4, 2017 at 10:50 AM, Stefan Richter < > >> s.rich...@data-artisans.com > >>> wrote: > >> > >>> +1 > >>> > >>> 1. Cluster tests > >>> > >>> Tested with the stateful state machine job on the following settings: > >>> - Cloud env: AWS > >>> - Distributions: EMR 5.7.0 > >>> - Flink deployment method: YARN (Hadoop 2.7.2) > >>> - HA: enabled > >>> - Kerberos: disabled > >>> - Kafka version: 0.10, 0.11 > >>> - State Backends: Heap (Sync / Async) & RocksDB (incremental / full) > >>> - Filesystem: S3 and HDFS (Hadoop 2.7.2) > >>> - Externalized checkpoints: enabled & disabled > >>> > >>> 2. Building with Scala 2.11 works > >>> > >>> 3. Building against Hadoop version works > >>> > >>>> Am 04.08.2017 um 08:00 schrieb Aljoscha Krettek <aljos...@apache.org > >: > >>>> > >>>> Hi everyone, > >>>> > >>>> Please review and vote on the release candidate #3 for the version > >>> 1.3.2, as follows: > >>>> [ ] +1, Approve the release > >>>> [ ] -1, Do not approve the release (please provide specific comments) > >>>> > >>>> > >>>> The complete staging area is available for your review, which > includes: > >>>> * JIRA release notes [1], > >>>> * the official Apache source release and binary convenience releases > to > >>> be deployed to dist.apache.org [2], which is signed with the key with > >>> fingerprint 0xA8F4FD97121D7293 [3], > >>>> * all artifacts to be deployed to the Maven Central Repository [4], > >>>> * source code tag "release-1.3.2-rc2" [5], > >>>> * website pull request listing the new release and adding announcement > >>> blog post [6]. > >>>> > >>>> The only change in this RC compared to the last is this commit that > >>> fixes copying of the Gelly examples jar: https://github.com/apache/ > >>> flink/commit/fda455e23a6192200b85b84d3aee312c9be40c99. I would > therefore > >>> like to propose a shorter voting period because the last RC was > seemingly > >>> good except for this bug. Please voice your concerns about this if you > >> have > >>> any. The vote will be open for at least 24 hours. It is adopted by > >> majority > >>> approval, with at least 3 PMC affirmative votes. > >>>> > >>>> Please use the provided document, as discussed before, for > coordinating > >>> the testing efforts: [7]. I have copied over the cluster testing > efforts > >>> and functional testing efforts since the code of this RC is exactly the > >>> same as RC2. > >>>> > >>>> Thanks, > >>>> Aljoscha > >>>> > >>>> [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa? > >>> projectId=12315522&version=12340984 > >>>> [2] http://people.apache.org/~aljoscha/flink-1.3.2-rc3/ > >>>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > >>>> [4] https://repository.apache.org/content/repositories/ > >>> orgapacheflink-1135/ > >>>> [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h= > >>> d0f7528a69a93045f4af686347753f8737ca0b2b > >>>> [6] https://github.com/apache/flink-web/pull/75 > >>>> [7] https://docs.google.com/document/d/1OJAE6scAZXbSBaGNNEpU1qhr3Jb7b > >>> wbec7keEnT9hmA/edit?usp=sharing > >>> > >>> > >> > >
