Hello, After lots of testing in local environments we're now trying to get our cluster running on AWS EMR. We followed much of the documentation from both AWS and Flink and have gotten to the point of creating a yarn session and submitting jobs. We successfully get back a Job ID and in the Yarn Timeline Server UI it says our application is running. However, we are having a hard time with logging.
2 main issues: 1. Logs for the jobmanager and taskmanager seem to take a long time to show up or in some cases just seem to never show up in the Yarn / Hadoop UI, even though we can see them just fine when ssh'ing into the cluster's nodes. Anything we can do to speed this up? 2. We can't seem to see anything except for WARN and ERROR logs for the jobmanager and taskmanager, we need at least INFO right now to confirm things are working as expected. We have been jumping through hoops going through a multitude of configuration files including log4j-session.properties and log4j.properties setting level to DEBUG but it has not helped. Are these the correct configuration files? Thanks! -- Rex Fenley | Software Engineer - Mobile and Backend Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | FOLLOW US <https://twitter.com/remindhq> | LIKE US <https://www.facebook.com/remindhq>
