Github user kl0u closed the pull request at:
https://github.com/apache/flink/pull/1984
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enable
Github user kl0u commented on the pull request:
https://github.com/apache/flink/pull/1984#issuecomment-219742461
Thanks for the comments @aljoscha .
The only comment not yet integrated is the one with the
{{OutputTypeConfigurable}} which I have to understand a bit better how
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63523040
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitReadOperator.java
---
@@ -0,0 +1,368 @@
+/*
+
Github user kl0u commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63521733
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitReadOperator.java
---
@@ -0,0 +1,368 @@
+/*
+ * Li
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1984#issuecomment-219673873
Overall, the code looks very good! I had some inline comments about
Javadoc/comments.
One thing that might be wrong, though is the interplay between
`Checkpoin
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63495352
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitReadOperator.java
---
@@ -0,0 +1,368 @@
+/*
+
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63494863
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitReadOperator.java
---
@@ -0,0 +1,368 @@
+/*
+
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63494540
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitReadOperator.java
---
@@ -0,0 +1,368 @@
+/*
+
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63493915
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitMonitoringFunction.java
---
@@ -0,0 +1,345 @@
+/*
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63493233
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitMonitoringFunction.java
---
@@ -0,0 +1,345 @@
+/*
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63492862
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitMonitoringFunction.java
---
@@ -0,0 +1,345 @@
+/*
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63492517
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitMonitoringFunction.java
---
@@ -0,0 +1,345 @@
+/*
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63492326
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitMonitoringFunction.java
---
@@ -0,0 +1,345 @@
+/*
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63492168
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FileSplitMonitoringFunction.java
---
@@ -0,0 +1,345 @@
+/*
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63491931
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/FilePathFilter.java
---
@@ -0,0 +1,74 @@
+/*
+ * Licens
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63491422
--- Diff:
flink-fs-tests/src/test/java/org/apache/flink/hdfstests/HDFSTest.java ---
@@ -115,7 +115,7 @@ public void testHDFS() {
}
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63491234
--- Diff:
flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/StreamExecutionEnvironment.scala
---
@@ -463,7 +463,7 @@ class StreamExe
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63490526
--- Diff:
flink-core/src/test/java/org/apache/flink/api/common/io/EnumerateNestedFilesTest.java
---
@@ -394,4 +394,4 @@ public IntValue nextRecord(IntValue
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63490499
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/io/FileInputFormat.java ---
@@ -235,7 +235,7 @@ protected FileInputFormat(Path filePath) {
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/1984#discussion_r63490061
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/io/BlockInfo.java ---
@@ -26,6 +26,11 @@
import org.apache.flink.core.memory.DataOutputVi
Github user kl0u commented on the pull request:
https://github.com/apache/flink/pull/1984#issuecomment-219117647
After the discussion we had today with @StephanEwen and @aljoscha , I also
added the PROCESS_ONCE watchType which processes the current (when invoked)
content of a file/dir
GitHub user kl0u opened a pull request:
https://github.com/apache/flink/pull/1984
[FLINK-3889] Make File Monitoring Function checkpointable.
This pull request introduces the underlying functionality to make Streaming
File Sources persistent.
It does not yet change the API calls
22 matches
Mail list logo
