[jira] [Created] (FLINK-7768) Load File Systems via Java Service abstraction

Stephan Ewen created FLINK-7768: --- Summary: Load File Systems via Java Service abstraction Key: FLINK-7768 URL: https://issues.apache.org/jira/browse/FLINK-7768 Project: Flink Issue Type: Improv

[jira] [Created] (FLINK-7767) Avoid loading Hadoop conf dynamically at runtime

Stephan Ewen created FLINK-7767: --- Summary: Avoid loading Hadoop conf dynamically at runtime Key: FLINK-7767 URL: https://issues.apache.org/jira/browse/FLINK-7767 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-7766) Remove obsolete reflection for hflush on HDFS

Stephan Ewen created FLINK-7766: --- Summary: Remove obsolete reflection for hflush on HDFS Key: FLINK-7766 URL: https://issues.apache.org/jira/browse/FLINK-7766 Project: Flink Issue Type: Improve

Re: System resource logger

System and processor info, marked as 'logged once' in gist shared by Piotr, should still be logged instead of registered as metrics, right? On Thu, Oct 5, 2017 at 2:38 AM, Till Rohrmann wrote: > Thanks for the proposal Piotr. I like it a lot since it will help people to > better understand their

Re: Unable to write snapshots to S3 on EMR

Hi Till, I believe this is what you are looking for, classpath is much bigger for the task manager. I can also post the whole log file if needed: 2017-10-05 14:17:53,038 INFO org.apache.flink.yarn.YarnTaskManagerRunner - Classpath: flink-consumer.jar:lib/flink-dist_2.11-1.3.2.

[jira] [Created] (FLINK-7765) Enable dependency convergence

Piotr Nowojski created FLINK-7765: - Summary: Enable dependency convergence Key: FLINK-7765 URL: https://issues.apache.org/jira/browse/FLINK-7765 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-7764) FlinkKafkaProducer010 does not accept name, uid, or parallelism

Fabian Hueske created FLINK-7764: Summary: FlinkKafkaProducer010 does not accept name, uid, or parallelism Key: FLINK-7764 URL: https://issues.apache.org/jira/browse/FLINK-7764 Project: Flink

[jira] [Created] (FLINK-7763) TableSinkITCase fails with "object reuse" enabled

Aljoscha Krettek created FLINK-7763: --- Summary: TableSinkITCase fails with "object reuse" enabled Key: FLINK-7763 URL: https://issues.apache.org/jira/browse/FLINK-7763 Project: Flink Issue T

Re: System resource logger

Thanks for the proposal Piotr. I like it a lot since it will help people to better understand their system. I would also be in favour of adding them to the system metrics. I think o.a.f.runtime.metrics.util.MetricUtils is the right place to start. Given the small dependency footprint and the compat

Re: Dependency convergence

+1 for pulling our dependencies straight and guarding it via the maven-enforcer-plugin. On Wed, Oct 4, 2017 at 8:58 PM, Piotr Nowojski wrote: > I meant for whole flink project. > > > On Oct 4, 2017, at 8:43 PM, Bowen Li wrote: > > > > +1. This is great, Piotrek! > > > > BTW, can you clarify wha

Re: Unable to write snapshots to S3 on EMR

Hi Andy, the CliFrontend is not executed via Yarn, thus, it is not affected by dependencies which are added due to the underlying Yarn cluster. Therefore, it would be helpful to look at the TaskManager logs. Either you have enabled log aggregation on your Yarn cluster, then you can obtain the logs

Re: System resource logger

+1 thanks for pointing this out. It makes sense to just expand those system metrics (I was not aware of them). > On Oct 4, 2017, at 6:07 PM, Greg Hogan wrote: > > What if we added these as system metrics and added a way to write metrics to > a (separate?) log file? > > >> On Oct 4, 2017, at

[jira] [Created] (FLINK-7762) Make WikipediaEditsSourceTest a proper test

Aljoscha Krettek created FLINK-7762: --- Summary: Make WikipediaEditsSourceTest a proper test Key: FLINK-7762 URL: https://issues.apache.org/jira/browse/FLINK-7762 Project: Flink Issue Type: I