Flink Benchmarks have been included in BigDataBench

2015-12-17 Thread Xinhui Tian
Hello everyone! Earlier we have posted a topic about the plan of including Flink support in our big data benchmark, which is called BigDataBench. The benchmarks can be found on http://prof.ict.ac.cn/BigDataBench/. Now we have released a new version including Flink benchmarks. It is now a beta ver

[jira] [Created] (FLINK-3184) Decrease Akka timeouts on cluster side to make system more responsive

2015-12-17 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-3184: Summary: Decrease Akka timeouts on cluster side to make system more responsive Key: FLINK-3184 URL: https://issues.apache.org/jira/browse/FLINK-3184 Project: Flink

Re: Flink shell in Jupyter

2015-12-17 Thread Tanya Schlusser
@Gyula if you want to delegate something small this interests me too. Best, Tanya On Thu, Dec 17, 2015 at 9:04 AM, Till Rohrmann wrote: > The only thing I did was to use the Py4J bridge to ship data between the > Scala and the Python process. That way, I could visualize data coming from > Flink

[jira] [Created] (FLINK-3183) Display direct memory stats in web frontend

2015-12-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3183: -- Summary: Display direct memory stats in web frontend Key: FLINK-3183 URL: https://issues.apache.org/jira/browse/FLINK-3183 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-3182) TaskManagersHandler can block web frontend

2015-12-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3182: -- Summary: TaskManagersHandler can block web frontend Key: FLINK-3182 URL: https://issues.apache.org/jira/browse/FLINK-3182 Project: Flink Issue Type: Bug

Re: Flink shell in Jupyter

2015-12-17 Thread Till Rohrmann
The only thing I did was to use the Py4J bridge to ship data between the Scala and the Python process. That way, I could visualize data coming from Flink with Python. I couldn't run Flink programs using the Python API, though. @Gyula, you can find the code here: https://github.com/tillrohrmann/inc

[jira] [Created] (FLINK-3181) The vertex-centric SSSP example and library method send unnecessary messages during the first superstep

2015-12-17 Thread Vasia Kalavri (JIRA)
Vasia Kalavri created FLINK-3181: Summary: The vertex-centric SSSP example and library method send unnecessary messages during the first superstep Key: FLINK-3181 URL: https://issues.apache.org/jira/browse/FLINK-3

Re: No job recovery after job manager failure

2015-12-17 Thread Ufuk Celebi
As an update: I’m investigating this. Ali sent me the log files. > On 16 Dec 2015, at 18:15, Ufuk Celebi wrote: > > Hey Ali, > > can you send me the complete logs? > > I don’t think it’s possible via the mailing list. Just send it to my private > email u...@apache.org. > > – Ufuk > >> On 16

Re: Flink shell in Jupyter

2015-12-17 Thread Gyula Fóra
That would be awesome, I think it would be fine to go for Scala 2.11 first in the notebook, if Python is more trouble. Gyula Stephan Ewen ezt írta (időpont: 2015. dec. 17., Cs, 14:17): > I think Till has done some advanced Pythin / Flink / Zeppelin integration > (to use Python plotting libs) fo

Re: Flink shell in Jupyter

2015-12-17 Thread Stephan Ewen
I think Till has done some advanced Pythin / Flink / Zeppelin integration (to use Python plotting libs) for a talk at some point. @Till: Do you still have the code? Could you share it with Gyula? On Wed, Dec 16, 2015 at 4:22 PM, Gyula Fóra wrote: > Hey Guys, > > Has anyone tried to setup the Fl

[jira] [Created] (FLINK-3180) MemoryLogger does not log direct memory

2015-12-17 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3180: -- Summary: MemoryLogger does not log direct memory Key: FLINK-3180 URL: https://issues.apache.org/jira/browse/FLINK-3180 Project: Flink Issue Type: Bug C

Re: [DISCUSS] Time Behavior in Streaming Jobs (Event-time/processing-time)

2015-12-17 Thread Maximilian Michels
Hi Aljoscha, I'm in favor of option 2: Keep the setStreamTimeCharacteristic to set the default time behavior. Then add a method to the operators to set a custom time behavior. The problem explanatory in SlidingTimeWindows: @Override public Trigger getDefaultTrigger(StreamExecutionEnvironment env

Re: [DISCUSS] Time Behavior in Streaming Jobs (Event-time/processing-time)

2015-12-17 Thread Kostas Tzoumas
Aljoscha, thanks for starting this discussion. I think this will be very important to get right. Can you explain a bit more why the results are "wrong"? I understand that window panes are built on event timestamps (as intended), but fired at regular intervals instead of watermarks. Why is this wro

Re: Apache Tinkerpop & Geode Integration?

2015-12-17 Thread James Thornton
Hello all - Yes, Cypher could run on the Gremlin Traversal Machine (GTM), and in some ways it already does. The GTM is like the JVM for graphs -- see this paper by Marko Rodriguez... - "The Gremlin Graph Traversal Machine and Language" http://arxiv.org/pdf/1508.03843v1.pdf And for a high-