Sorry, it was already out. I was merely struggling with the Maven deploy
command because "tools/generate_specific_pom.sh" is not entirely compatible
with old versions of Perl or sed. The script was generating incorrect
pom.xml files..
On Sat, Jun 13, 2015 at 9:14 PM, Aljoscha Krettek
wrote:
> Th
The new release candidate is not yet done? We have a very simple fix that
allows the RowSerializer of the Table API to work with null-fields. I think
we should include that. What do you think?
On Fri, 12 Jun 2015 at 23:50 Ufuk Celebi wrote:
> I'm with Till on this. Robert's position is valid as
I'm with Till on this. Robert's position is valid as well. Again, there is
no core disagreement here. No one wants to add it to dist.
On 12 Jun 2015, at 00:40, Ufuk Celebi > wrote:
On 11 Jun 2015, at 20:04, Fabian Hueske > wrote:
How about the following issues?
1. The Hbase Hadoop Compat issue
I agree mostly with Robert. However, one could also argue that by not
including the libraries in the dist package, the user code jar will also be
blown up by the dependencies added by the library. This will slow down job
submission, because it has to be distributed on the cluster. Furthermore, I
wo
Regarding the discussion with including ML, Gelly, streaming connectors
into "flink-dist".
I'm strongly against adding those into our jar because they blow up the
dependencies we are shipping by default.
Also, the maven archetype sets up everything so that the dependencies are
packaged into the us
I've finished the legal check of the source and binary distribution. The PR
with the LICENSE and NOTICE file updates can be found here [1].
What I haven't done yet is addressing the issue with the shaded
dependencies. I think that we have to add to all jars which contain
dependencies as binary dat
I almost finished creating the new release candidate. Then the maven deploy
command failed on me for the hadoop1 profile:
[INFO]
[INFO] BUILD FAILURE
[INFO]
OK, guys. I merged and pushed the last outstanding commits to the
release-0.9 branch.
Good to go for a new candidate.
2015-06-12 14:30 GMT+02:00 Maximilian Michels :
> +1 Let's constitute the changes in a new release candidate.
>
> On Fri, Jun 12, 2015 at 2:06 PM, Fabian Hueske wrote:
>
> > +1 f
+1 Let's constitute the changes in a new release candidate.
On Fri, Jun 12, 2015 at 2:06 PM, Fabian Hueske wrote:
> +1 for b)
>
> I'm organizing + merging the commits that need to go the new candidate
> right now. Will let you know, when I am done.
>
> 2015-06-12 14:03 GMT+02:00 Till Rohrmann :
+1 for b)
I'm organizing + merging the commits that need to go the new candidate
right now. Will let you know, when I am done.
2015-06-12 14:03 GMT+02:00 Till Rohrmann :
> I'm in favour of option b) as well.
>
> On Fri, Jun 12, 2015 at 12:05 PM Ufuk Celebi wrote:
>
> > Yes, the LICENSE files ar
I'm in favour of option b) as well.
On Fri, Jun 12, 2015 at 12:05 PM Ufuk Celebi wrote:
> Yes, the LICENSE files are definitely a release blocker.
>
> a) Either we wait with the RC until we have fixed the LICENSES, or
>
> b) Put out next RC to continue with testing and then update it with the
>
Yes, the LICENSE files are definitely a release blocker.
a) Either we wait with the RC until we have fixed the LICENSES, or
b) Put out next RC to continue with testing and then update it with the
LICENSE [either we find something before the LICENSE update or we only have
to review the LICENSE cha
What about the shaded jars?
On Fri, Jun 12, 2015 at 11:32 AM Ufuk Celebi wrote:
> @Max: for the new RC. Can you make sure to set the variables correctly
> with regard to stable/snapshot versions in the docs?
@Max: for the new RC. Can you make sure to set the variables correctly with
regard to stable/snapshot versions in the docs?
On 12 Jun 2015, at 00:40, Ufuk Celebi wrote:
>
> On 11 Jun 2015, at 20:04, Fabian Hueske wrote:
>
>> How about the following issues?
>>
>> 1. The Hbase Hadoop Compat issue, Ufuk is working on
>
> I was not able to reproduce this :( I ran HadoopInputFormats against various
> sources and con
After thinking about it a bit more, I think that's fine.
+1 to document and keep it as it is.
Just to clarify. If you write a Flink program and include the Table API as
a dependency, then you have to package your program in the JAR with the
Table API and submit it to the cluster. IMHO that's ok but it should be
documented to inform users which libraries are included in Flink binaries
out of
On 12 Jun 2015, at 10:44, Till Rohrmann wrote:
> Yes you're right Ufuk. At the moment the user has to place the jars in the
> lib folder of Flink. If this folder is not shared then he has to do it for
> every node on which Flink runs.
OK. I guess there is a nice way to do this with YARN as well
Yes you're right Ufuk. At the moment the user has to place the jars in the
lib folder of Flink. If this folder is not shared then he has to do it for
every node on which Flink runs.
On Fri, Jun 12, 2015 at 10:42 AM Till Rohrmann wrote:
> I think I found a real release blocker. Currently we don't
I think I found a real release blocker. Currently we don't add license
files to our shaded jars. For example
the flink-shaded-include-yarn-0.9.0-milestone-1.jar shades hadoop code.
This code also includes the `org.apache.util.bloom.*` classes. These
classes are licensed under The European Commissi
On 12 Jun 2015, at 10:29, Till Rohrmann wrote:
> Well I think the initial idea was to keep the dist jar as small a possible
> and therefore we did not include the libraries. I'm not sure whether we can
> decide this here ad-hoc. If the community says that we shall include these
> libraries then
Well I think the initial idea was to keep the dist jar as small a possible
and therefore we did not include the libraries. I'm not sure whether we can
decide this here ad-hoc. If the community says that we shall include these
libraries then I can add them. But bear in mind that all of them have som
As for outstanding issues I think streaming is good to go as far as I know.
I am personally against including all libraries - at least speaking for the
streaming connectors. Robert, Stephan and myself had a detailed discussion
on that some time ago and the disadvantage of having all the libraries i
I have another fix, but this is just a documentation update (FLINK-2207)
and will be done soon.
2015-06-12 10:02 GMT+02:00 Maximilian Michels :
> We should have a nightly cluster test for every library. Let's keep that in
> mind for the future. Very nice find, Till!
>
> Since there were not objec
We should have a nightly cluster test for every library. Let's keep that in
mind for the future. Very nice find, Till!
Since there were not objections, I cherry-picked the proposed commits from
the document to the release-0.9 branch. If I understand correctly, we can
create the new release candida
On 12 Jun 2015, at 09:45, Till Rohrmann wrote:
> Hi guys,
>
> I just noticed while testing the TableAPI on the cluster that it is not
> part of the dist module. Therefore, programs using the TableAPI will only
> run when you put the TableAPI jar directly on the cluster or if you build a
> fat j
@Till: This also apples to the streaming connectors.
On Fri, Jun 12, 2015 at 9:45 AM, Till Rohrmann wrote:
> Hi guys,
>
> I just noticed while testing the TableAPI on the cluster that it is not
> part of the dist module. Therefore, programs using the TableAPI will only
> run when you put the Tab
Hi guys,
I just noticed while testing the TableAPI on the cluster that it is not
part of the dist module. Therefore, programs using the TableAPI will only
run when you put the TableAPI jar directly on the cluster or if you build a
fat jar including the TableAPI jar. This is nowhere documented.
Fur
I'm currently going through the license file and I discovered some
skeletons in our closet. This has to be merged as well. But I'm still
working on it (we have a lot of dependencies).
Cheers,
Till
On Fri, Jun 12, 2015 at 12:51 AM Ufuk Celebi wrote:
>
> On 12 Jun 2015, at 00:49, Fabian Hueske w
On 12 Jun 2015, at 00:49, Fabian Hueske wrote:
> 2. is basically done. I have a patch which updates the counters on page
> reload but that shouldn't be hard to extend to dynamic updates.
Very nice! :-) Thanks!
2. is basically done. I have a patch which updates the counters on page
reload but that shouldn't be hard to extend to dynamic updates.
2015-06-12 0:40 GMT+02:00 Ufuk Celebi :
>
> On 11 Jun 2015, at 20:04, Fabian Hueske wrote:
>
> > How about the following issues?
> >
> > 1. The Hbase Hadoop Com
On 11 Jun 2015, at 20:04, Fabian Hueske wrote:
> How about the following issues?
>
> 1. The Hbase Hadoop Compat issue, Ufuk is working on
I was not able to reproduce this :( I ran HadoopInputFormats against various
sources and confirmed the results and everything was fine so far.
I think I w
How about the following issues?
1. The Hbase Hadoop Compat issue, Ufuk is working on
2. The incorrect webinterface counts
@Ufuk were you able to reproduce the bug?
The deadlock in the scheduler is now fixed. Based on the changes that have
been push to the release-0.9 branch, I'd like to create a
Yes, we would include those in the new release candidate.
On Jun 11, 2015 5:22 PM, "Aljoscha Krettek" wrote:
> Aren't there still some commits at the top of the release document that
> need to be cherry-picked to the release branch?
>
> On Thu, 11 Jun 2015 at 17:13 Maximilian Michels wrote:
>
>
Aren't there still some commits at the top of the release document that
need to be cherry-picked to the release branch?
On Thu, 11 Jun 2015 at 17:13 Maximilian Michels wrote:
> The deadlock in the scheduler is now fixed. Based on the changes that have
> been push to the release-0.9 branch, I'd l
The deadlock in the scheduler is now fixed. Based on the changes that have
been push to the release-0.9 branch, I'd like to create a new release
candidate later on. I think we have gotten the most critical issues out of
the way. Would that be ok for you?
On Wed, Jun 10, 2015 at 5:56 PM, Fabian Hue
Yes, that needs to be fixed IMO
2015-06-10 17:51 GMT+02:00 Till Rohrmann :
> Yes since it is clearly a deadlock in the scheduler, the current version
> shouldn't be released.
>
> On Wed, Jun 10, 2015 at 5:48 PM Ufuk Celebi wrote:
>
> >
> > On 10 Jun 2015, at 16:18, Maximilian Michels wrote:
> >
Yes since it is clearly a deadlock in the scheduler, the current version
shouldn't be released.
On Wed, Jun 10, 2015 at 5:48 PM Ufuk Celebi wrote:
>
> On 10 Jun 2015, at 16:18, Maximilian Michels wrote:
>
> > I'm debugging the TaskManagerFailsWithSlotSharingITCase. I've located its
> > cause bu
On 10 Jun 2015, at 16:18, Maximilian Michels wrote:
> I'm debugging the TaskManagerFailsWithSlotSharingITCase. I've located its
> cause but still need to find out how to fix it.
Very good find, Max!
Max, Till, and I have looked into this and it is a reproducible deadlock in the
scheduler duri
I'm debugging the TaskManagerFailsWithSlotSharingITCase. I've located its
cause but still need to find out how to fix it.
On Wed, Jun 10, 2015 at 2:25 PM, Aljoscha Krettek
wrote:
> I added a section at the top of the release testing document to keep
> track of commits that we might want to cherr
I added a section at the top of the release testing document to keep
track of commits that we might want to cherry-pick to the release. I
included the YARNSessionFIFOITCase fix and the optional stream
iteration partitioning (both already on release branch).
On Wed, Jun 10, 2015 at 12:51 PM, Fabian
@Sachin: I reproduced the build error on my Windows machine.
2015-06-10 12:22 GMT+02:00 Maximilian Michels :
> @Sachin: This looks like a file permission issue. We should have someone
> else verify that on a Windows system.
>
> On Wed, Jun 10, 2015 at 11:28 AM, Sachin Goel
> wrote:
>
> > I have
@Sachin: This looks like a file permission issue. We should have someone
else verify that on a Windows system.
On Wed, Jun 10, 2015 at 11:28 AM, Sachin Goel
wrote:
> I have run "mvn clean verify" five times now and every time I'm getting
> these failed tests:
>
> BlobUtilsTest.before:45 null
>
Regarding the iteration partitioning feature, since I use it of course I
find it very useful, but it is true that it needs to be tested more
extensively and also be discussed by the community before it is added in a
release.
Moreover, given the fact that I can still use it for research purposes (I
I have run "mvn clean verify" five times now and every time I'm getting
these failed tests:
BlobUtilsTest.before:45 null
BlobUtilsTest.before:45 null
BlobServerDeleteTest.testDeleteFails:291 null
BlobLibraryCacheManagerTest.testRegisterAndDownload:196 Could not
remove write permissions from
The KMeans quickstart example does not work with the current state of
the KMeansDataGenerator. I created PR that brings the two in sync.
This should probably go into the release since it affects initial user
"satisfaction".
On Wed, Jun 10, 2015 at 11:14 AM, Márton Balassi
wrote:
> As for the stre
As for the streaming commit cherry-picked to the release branch:
This is an unfortunate communication issue, let us make sure that we
clearly communicate similar issues in the future.
As for FLINK-2192: This is essentially a duplicate issue of the testability
of the streaming iteration. Not a bloc
Hey Gyula, Max,
On 10 Jun 2015, at 10:54, Gyula Fóra wrote:
> This feature needs to be included in the release, it has been tested and
> used extensively. And many applciations depend on it.
It would be nice to announce/discuss this before just cherry-picking it into
the release branch. The is
Adding one more thing to the list:
The code contains a misplaced class (mea culpa) in flink-java,
org.apache.flink.api.java.SortPartitionOperator which is API facing and
should be moved to the operators package. If we do that after the release,
it will break binary compatibility. I created FLINK-2
I'm not against including the feature but I'd like to discuss it first. I
believe that only very carefully selected commits should be added to
release-0.9. If that feature happens to be tested extensively and is very
important for user satisfactory then we might include it.
On Wed, Jun 10, 2015 at
I agree with Gyula regarding the iteration partitioning.
I have also been using this feature for developing machine learning
algorithms. And I think SAMOA also needs this feature.
Faye
2015-06-10 10:54 GMT+02:00 Gyula Fóra :
> This feature needs to be included in the release, it has been tested
This feature needs to be included in the release, it has been tested and
used extensively. And many applciations depend on it.
Maximilian Michels ezt írta (időpont: 2015. jún. 10., Sze,
10:47):
> With all the issues discovered, it looks like we'll have another release
> candidate. Right now, we
With all the issues discovered, it looks like we'll have another release
candidate. Right now, we have discovered the following problems:
1 YARN ITCase fails [fixed via 2eb5cfe]
2 No Jar for SessionWindowing example [fixed in #809]
3 Wrong description of the input format for the graph examples (eg
This doesn't look good, yes.
On Wed, Jun 10, 2015 at 1:32 AM, Ufuk Celebi wrote:
> While looking into FLINK-2188 (HBase input) I've discovered that Hadoop
> input formats implementing Configurable (like mapreduce.TableInputFormat)
> don't have the Hadoop configuration set via setConf(Configurati
While looking into FLINK-2188 (HBase input) I've discovered that Hadoop input
formats implementing Configurable (like mapreduce.TableInputFormat) don't have
the Hadoop configuration set via setConf(Configuration).
I have a small fix for this, which I have to clean up. First, I wanted to check
w
I attached jps and jstack log about hanging
TaskManagerFailsWithSlotSharingITCase to JIRA FLINK-2183.
Regards,
Chiwan Park
> On Jun 10, 2015, at 12:28 AM, Aljoscha Krettek wrote:
>
> I discovered something that might be a feature, rather than a bug. When you
> submit an example using the web c
I discovered something that might be a feature, rather than a bug. When you
submit an example using the web client without giving parameters the
program fails with this:
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error.
at
org.apache.flink.client.program
I discovered another problem:
https://issues.apache.org/jira/browse/FLINK-2191 The closure cleaner
cannot be disabled in part of the Streaming Java API and all of the
Streaming Scala API. I think this is a release blocker (in addition
the the other bugs found so far.)
On Tue, Jun 9, 2015 at 2:35 P
I found the bug in the failing YARNSessionFIFOITCase: It was comparing
the hostname to a hostname in some yarn config. In one case it was
capitalised, in the other case it wasn't.
Pushing fix to master and release-0.9 branch.
On Tue, Jun 9, 2015 at 2:18 PM, Sachin Goel wrote:
> A re-ran lead to
A re-ran lead to reproducibility of 11 failures again.
TaskManagerTest.testSubmitAndExecuteTask was failing with a time-out but
managed to succeed in a re-run. Here is the log output again:
http://pastebin.com/raw.php?i=N4cm1J18
Setup: JDK 1.8.0_40 on windows 8.1
System memory: 8GB, quad-core with
On 09 Jun 2015, at 13:58, Sachin Goel wrote:
> On my local machine, several flink runtime tests are failing on "mvn clean
> verify". Here is the log output: http://pastebin.com/raw.php?i=VWbx2ppf
Thanks for reporting this. Have you tried it multiple times? Is it failing
reproducibly with the s
I did five "mvn clean verify" runs by now. All of them failed. One
with the TaskmanagerFailsWithSlotSharingITCase and the other ones with
YARNSessionFIFOITCase
On Tue, Jun 9, 2015 at 12:34 PM, Aljoscha Krettek wrote:
> I also saw the same error on my third "mvn clean verify" run. Before it
> alwa
On my local machine, several flink runtime tests are failing on "mvn clean
verify". Here is the log output: http://pastebin.com/raw.php?i=VWbx2ppf
--
Sachin
On Tue, Jun 9, 2015 at 4:04 PM, Aljoscha Krettek
wrote:
> I also saw the same error on my third "mvn clean verify" run. Before it
> alw
I also saw the same error on my third "mvn clean verify" run. Before it
always failed in the YARN tests.
On Tue, Jun 9, 2015 at 12:23 PM, Till Rohrmann wrote:
> I also encountered a failing TaskManagerFailsWithSlotSharingITCase using
> Java8. I could, however, not reproduce the error a second ti
I also encountered a failing TaskManagerFailsWithSlotSharingITCase using
Java8. I could, however, not reproduce the error a second time. The stack
trace is:
The JobManager should handle hard failing task manager with slot
sharing(org.apache.flink.runtime.jobmanager.TaskManagerFailsWithSlotSharingI
The name of the Git branch was not correct. Thank you, Aljoscha, for
noticing. I've changed it from "release-0.9-rc1" to "release-0.9.0-rc1".
This has no affect on the validity of the release candidate.
+1 makes sense.
On Tue, Jun 9, 2015 at 10:48 AM, Aljoscha Krettek
wrote:
> I would suggest we use this format to notify others that we did a task:
>
> Assignees:
> - Aljoscha: done
> - Ufuk: found bug in such an such...
> - Chiwan Park: done, ...
>
> The simple "status" doesn't work with mult
I would suggest we use this format to notify others that we did a task:
Assignees:
- Aljoscha: done
- Ufuk: found bug in such an such...
- Chiwan Park: done, ...
The simple "status" doesn't work with multiple people on one task.
On Tue, Jun 9, 2015 at 9:40 AM, Ufuk Celebi wrote:
> Hey all,
>
Hey all,
1. it would be nice if we find more people to also do testing of the streaming
API. I think it's especially good to have people on it, which did not use it
before.
2. Just to make sure: the "assignee" field of each task is a list, i.e. we can
and should have more people testing per ta
Hey Chiwan!
Is the problem reproducible? Does it always deadlock? Can you please wait
for it to deadlock and then post a stacktrace (jps and jstack) of the
process? Please post it to this issue: FLINK-2183.
Thanks :)
– Ufuk
On Monday, June 8, 2015, Chiwan Park > wrote:
> Hi. I have a problem r
Hi. I have a problem running `mvn clean verify` command.
TaskManagerFailsWithSlotSharingITCase hangs in Oracle JDK 7 (1.7.0_80). But in
Oracle JDK 8 the test case doesn’t hang.
I’ve investigated about this problem but I cannot found the bug.
Regards,
Chiwan Park
> On Jun 9, 2015, at 2:11 AM, Má
Added F7 Running against Kafka cluster for me in the doc. Doing it
tomorrow.
On Mon, Jun 8, 2015 at 7:00 PM, Chiwan Park wrote:
> Hi. I’m very excited about preparing a new major release. :)
> I just picked two tests. I will report status as soon as possible.
>
> Regards,
> Chiwan Park
>
> > On
Hi. I’m very excited about preparing a new major release. :)
I just picked two tests. I will report status as soon as possible.
Regards,
Chiwan Park
> On Jun 9, 2015, at 1:52 AM, Maximilian Michels wrote:
>
> Hi everyone!
>
> As previously discussed, the Flink developer community is very eager
Hi everyone!
As previously discussed, the Flink developer community is very eager to get
out a new major release. Apache Flink 0.9.0 will contain lots of new
features and many bugfixes. This time, I'll try to coordinate the release
process. Feel free to correct me if I'm doing something wrong beca
74 matches
Mail list logo
