Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5966
@StephanEwen I would like to work on this issue, building on your PR, would
that be OK?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5966
To be honest I don't see a great need to support anything other than a
single trusted certificate for internal communication.We could easily build
some conveniences around that,
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5966
This looks great but let's please separate the intra-cluster SSL
configuration options from the external-cluster options. In particular, the
web/API endpoints should be governed by a diff
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5896
This PR seems to obfuscate the fix. The issue was with the interpretation
of the keytab path, right? But the bulk of the change was to clarify the
ordering of context installation vs use?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5427
@StephanEwen thanks again for the feedback, which I took to heart and
simplified the hook. It now has a `reset ` method that is called only in the
special case.
I will refactor the
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5427
@tillrohrmann @StephanEwen sorry about the long delay here, would you
please take another look?
I followed Stephan's suggestion of not introducing a new method. However,
the sema
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4767
@tillrohrmann do you need a websocket yet?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5634
I think the ideal would be that idleness would occur only for tail reads,
i.e. due to a timeout from `kafkaConsumer.poll(pollTimeout)`.In other
words, an intermittent connection issue would
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5427
I feel that we're not addressing the core issue that we're trying to fix.
1. A new job starts up with checkpointing enabled and a hook-based source.
2. The source begins
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5427
@StephanEwen do you mean that we could avoid adding an initialization
method to the hook interface? We need to somehow call into the hook to reset
its state, even when there's no checkpoint
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5427
@StephanEwen thanks for taking a look, I agree with trying to avoid a new
lifecycle method.
The `initializeState` method on the hook interface gives the hook an
unconditional
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5427
@tzulitai @StephanEwen please take a look, thanks.
---
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/5427
[FLINK-8533] [checkpointing] Support MasterTriggerRestoreHook state
reinitialization
Signed-off-by: Eron Wright
## What is the purpose of the change
Support
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5307
@tillrohrmann regarding your general question, you are right that Flink
could, in concept, deploy to GPU hosts even if Flink doesn't require any GPUs.
But we should keep in mind the inte
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r164529293
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
---
@@ -238,6 +254,12 @@ public
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4510
@tzulitai is anything left to do here?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5346
+1
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5307
@tillrohrmann please merge
---
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r162466070
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
---
@@ -238,6 +254,16 @@ public
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r162464500
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosResourceManager.java
---
@@ -661,6 +661,7 @@ private
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r162463088
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java
---
@@ -174,8 +183,9 @@ public
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r162462667
--- Diff: flink-mesos/pom.xml ---
@@ -88,7 +88,7 @@ under the License.
com.netflix.fenzo
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r162473916
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/scheduler/Offer.java ---
@@ -81,11 +82,14 @@ public Offer(Protos.Offer offer
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5307#discussion_r162468156
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/scheduler/Offer.java ---
@@ -156,6 +164,16 @@ public long getOfferedTime
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5172#discussion_r161822214
--- Diff: flink-yarn/src/test/resources/flink-conf.yaml ---
@@ -0,0 +1,23
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5172#discussion_r161822100
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnTaskManagerRunner.java ---
@@ -142,19 +134,22 @@ public static void runYarnTaskManager
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5172#discussion_r161821092
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnTaskManagerRunner.java ---
@@ -126,12 +124,6 @@ public static void runYarnTaskManager(String
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5172#discussion_r161818655
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/security/SecurityUtils.java
---
@@ -48,7 +48,7 @@ public static SecurityContext
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4767
@tillrohrmann are you still interested in this websocket code for the REST
server? Aside from rebasing, any 'must fix' issues here?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5208
@aljoscha that might work but the independence is also nice. In this case,
Fenzo is using Jackson 2.4.5 while Flink itself is using 2.7.9. How strongly
do you feel about this?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5208
@aljoscha please review
---
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/5208
[FLINK-8265] [Mesos] Missing jackson dependency for flink-mesos
## What is the purpose of the change
Fix for missing dependency in Mesos deployments.
## Brief change log
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5190#discussion_r158545462
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java
---
@@ -155,6 +156,7 @@ protected void runCluster
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5190
The configuration option `REST_ADDRESS` seems destined to cause pain.
Surveying the code, it seems to variously represent a bind address, a server
address, and an advertised address. We should
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5172
This PR probably fixes the problem, but it would be good to address the
deeper problem that the code is confusing. At least we could add some
commentary to the code. The specific problems, in
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5176
@NicoK was this PR a response to an actual issue, and did it resolve it?
Do I understand correctly that this will cause the JM (blob server) to use
`taskmanager.tmp.dirs`? In addition
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5176#discussion_r157548338
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobUtils.java ---
@@ -127,21 +132,31 @@ private static BlobStoreService
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5114
@tillrohrmann should we patch the 1.3 release too? Or is 1.3.2 the last
release there?
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5114
@tillrohrmann I believe your feedback has been addressed, please take a
look thanks.
---
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5114#discussion_r155026726
--- Diff:
flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala
---
@@ -185,7 +189,7 @@ class LaunchCoordinator
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5114#discussion_r155000484
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java
---
@@ -217,15 +222,13 @@ public String
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5114#discussion_r154999162
--- Diff:
flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala
---
@@ -148,14 +149,17 @@ class LaunchCoordinator
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/5114#discussion_r154998531
--- Diff:
flink-mesos/src/main/scala/org/apache/flink/mesos/scheduler/LaunchCoordinator.scala
---
@@ -148,14 +149,17 @@ class LaunchCoordinator
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/5114
Log output showing a 'mixed' scenario, involving two tasks, combining
resources across the reserved and unreserved resources in a single offer.
```
Assigning resources: [cpus(m
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/5114
[FLINK-8174] [mesos] Mesos RM unable to accept offers for unreserved
resources
## What is the purpose of the change
This PR improves the Mesos RM to work correctly with both reserved and
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4767
I don't much like the use of `RequestBody` and `ResponseBody` here, or even
that the WebSocket distinguishes between client and server messages. Honestly
a `MessageBody` marker interfac
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4767#discussion_r144381682
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/AbstractWebSocketHandler.java
---
@@ -0,0 +1,301 @@
+/*
+ * Licensed
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4807
Thanks for doing this, we need Akka 2.4 badly for improved SSL support.
---
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144320740
--- Diff: docs/start/building.md ---
@@ -115,19 +115,7 @@ The `-Pvendor-repos` activates a Maven [build
profile](http://maven.apache.org/g
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4767
Updated the description based on the latest PR.
---
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4765#discussion_r142713741
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/util/HandlerUtils.java
---
@@ -108,19 +108,25 @@ public static void
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4767#discussion_r142712872
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/websocket/WebSocket.java
---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4767#discussion_r142712091
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/rest/RestEndpointITCase.java
---
@@ -325,6 +406,98 @@ public TestParameters
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4767#discussion_r142700416
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java ---
@@ -275,4 +301,121 @@ public HttpResponseStatus
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4767#discussion_r142696121
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/AbstractRestHandler.java
---
@@ -157,7 +160,12 @@ protected void
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/4767
[FLINK-7738] [flip-6] Create WebSocket handler (server, client)
## What is the purpose of the change
Introduces WebSocket support for the FLIP-6 REST server and client.
The
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/4766
[FLINK-7752] [flip-6] RedirectHandler should execute on the IO thread
## What is the purpose of the change
Improves the determinism of REST handlers by running the response method
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/4765
[FLINK-7753] [flip-6] close REST channel on server error
## What is the purpose of the change
Improve REST server handling of unexpected errors.
## Brief change log
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3061
@VanessaHenderson nothing at this time.If you're not able to use
hostname verification as-is (given the non-canonical names that are used),
consider carefully controlling the truststore
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4616
+1
---
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r140108752
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -31,10 +31,50 @@
public
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4628#discussion_r140051191
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosFlinkResourceManager.java
---
@@ -663,6 +666,7 @@ private void
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4628#discussion_r140021958
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java
---
@@ -366,6 +425,34 @@ public
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4628#discussion_r140022455
--- Diff:
flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/LaunchableMesosWorker.java
---
@@ -155,7 +155,7 @@ public int
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4628#discussion_r140038861
--- Diff: docs/ops/deployment/mesos.md ---
@@ -226,6 +226,10 @@ When running Flink with Marathon, the whole Flink
cluster including the job mana
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4622
@tillrohrmann please take a look and merge if you're OK with it.
---
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4622
@bbayani thanks very much for this contribution. Please confirm my
understanding of the issue and verification. It appears that the resources
that are requested via `TaskInfo` must reference
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4510#discussion_r137395149
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/executiongraph/ExecutionGraphRescalingTest.java
---
@@ -0,0 +1,235
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r137082133
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -35,6 +35,39
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4636
In certain environments today, the Hadoop dependencies are actually
duplicated on the classpath, i.e. in the Flink dist jar and also via the local
installed Hadoop (when something like
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4636
Overall this looks great. I looked for any signs of behavior changes,
especially in the security code since there's some subtlety there, didn't
notice any.
Please consi
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r136615982
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -35,6 +35,39
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r136615089
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -35,6 +35,39
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r136614448
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -35,6 +35,39
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4415
There is only one true fix, that is to propagate the configuration instance
everywhere so that only the entrypoint calls loadConfiguration.
---
If your project is set up for it, you can reply to
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4415
@tillrohrmann @zentol the whole reason for the static field is to allow the
various sites that call GlobalConfiguration.loadConfiguration to obtain the
dynamic properties.Any property could
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4560
@tillrohrmann fixed the test issue and other noted issues. Once #4555 is
in, this is good to go.
The test issue was a race in RM acquiring leadership vs RPC call. Added
some code to
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4560#discussion_r134004745
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/slotmanager/TaskManagerRegistration.java
---
@@ -68,7 +68,9 @@ public
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4560#discussion_r134003593
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManager.java
---
@@ -960,12 +965,26 @@ public void handleError
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/4560
Flink 7077
## Description
_This PR extends #4555 - please disregard commits from Aug 16, 2017._
[FLINK-7077] [mesos] Implement task release to support dynamic scaling
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/4555
[FLINK-6630] [Mesos] Implement FLIP-6 MesosAppMasterRunner
## What is the purpose of the change
Implement Mesos runners for FLIP-6.
[FLINK-6630] [Mesos] Implement FLIP-6
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/2425
Note to future self: to generate a self-signed certificate, use
`CertAndKeyGen` and see
[OPENDJ-2247](https://bugster.forgerock.org/jira/browse/OPENDJ-2247).
---
If your project is set up for
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4368
@pnowojski looks great. You mentioned the Pravaga connector as a
motivation, did you look at its implementation and do you anticipate any
challenges in porting it o this new framework
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4281#discussion_r128130195
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/entrypoint/YarnJobClusterEntrypoint.java
---
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4281#discussion_r128129505
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/entrypoint/YarnEntrypointUtils.java
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4281#discussion_r128129161
--- Diff:
flink-yarn/src/main/java/org/apache/flink/yarn/entrypoint/YarnEntrypointUtils.java
---
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4261
Looking at the various diagrams of FLIP-6, there's quite a few possible
arrangements of the 'building blocks'. I wonder if composition over
inheritance would be preferable as
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4261#discussion_r128109508
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/entrypoint/ClusterEntrypoint.java
---
@@ -0,0 +1,247 @@
+/*
+ * Licensed to the
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4260
My take on this skeleton is that the `Dispatcher` should not use the
`JobManagerRunner` directly. My rationale is:
1. The dispatcher should never execute user code, otherwise it cannot
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3356
@tillrohrmann the `dynamicProperties` static field is unfortunate, yes.
To eliminate it, we must evaluate the use of the zero-arg
`GlobalConfiguration.loadConfiguration()` that is used in many
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3942
@tillrohrmann thoughts on this? I'm open to anything.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project doe
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4024
Would you mind remarking as to whether it is safe to delete a savepoint
after resuming a job from it?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3776
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3776
@Rucongzhang thanks for the contribution. I think I understand the problem
and your solution, which I will recap. I also found YARN-2704 to be useful
background.
*Problem*:
1. YARN
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/3987#discussion_r118530090
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/DefaultOperatorStateBackend.java
---
@@ -185,13 +185,18 @@ public void dispose
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3942
This is a fairly straightforward port of the RM code to the FLIP-6
architecture.
Follow-ups to occur in future PRs include:
- adding support for releasing the worker
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/3942
FLINK-6379 Mesos ResourceManager (FLIP-6)
- Make the RPC gateway of the ResourceManager extensible to allow for
framework-specific RPC methods
- Introduce FLIP-6 MesosResourceManager w
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/3933#discussion_r117370390
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/hooks/MasterHooks.java
---
@@ -267,6 +269,112 @@ else if (!allowUnmatchedState
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3933
Originally I pursued the path suggested by @tillrohrmann of setting the
TCCL at the sites where the hook is invoked, but found that the classloader
wasn't readily available.I found it
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/3933
That's the sort of workaround that Android apps are forced to use. Why
not continue the practice of setting the TCCL in transitions from system to
user code?
---
If your project is set u
1 - 100 of 209 matches
Mail list logo
