Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1090#issuecomment-137499875 Thanks for pointing me to the `distcp` page. So far, I was agnostic of this tool :) The performance difference between Hadoop and Flink should not be too different because the copying of files is mostly IO-bound work. Still, it is 1.5 minutes faster. Not sure if we can include your code in the Flink examples but definitely under `flink-contrib` where we usually put external tools that are not directly part of Flink.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---