Re: API request to submit job takes over 1hr

2016-06-15 Thread Robert Metzger
Hi, Regarding Shannon first point: I agree. We can improve the user experience a lot, and documenting the behavior is the first step we should do here. I see your points. I agree that we should use a separate thread for running the main method and report better to the front end what's happening.

Re: API request to submit job takes over 1hr

2016-06-13 Thread Tzu-Li (Gordon) Tai
Hi Shannon, Thanks for your investigation on the issue and the JIRA. There's actually a previous JIRA on this problem already: https://issues.apache.org/jira/browse/FLINK-4023. Would you be ok with tracking this issue on FLINK-4023, and close FLINK-4069 as a duplicate issue? As you can see, I've a

Re: API request to submit job takes over 1hr

2016-06-13 Thread Shannon Carey
t;> Date: Thursday, June 2, 2016 at 4:19 AM To: "user@flink.apache.org<mailto:user@flink.apache.org>" mailto:user@flink.apache.org>> Subject: Re: API request to submit job takes over 1hr Hi Shannon, thank you for further investigating the issue. Its fine to keep the disc

Re: API request to submit job takes over 1hr

2016-06-02 Thread Robert Metzger
Hi Shannon, thank you for further investigating the issue. Its fine to keep the discussion on the user@ list. Most devs are on the user list as well and we'll probably file some JIRAs. Regarding your suggestions: 1. Not sure if making the job submission non-blocking is a good idea. We would proba

Re: API request to submit job takes over 1hr

2016-06-01 Thread Shannon Carey
It looks like the problem is due to the stack trace below. Simply put, connection failure to Kafka when using the default settings causes job submission to take over (flink.get-partitions.retry * tries by SimpleConsumer * socket.timeout.ms * # of Kafka brokers) = (3 * 2 * 30 * (# of Kafka broke