Hi all,
I’m running this command, on the master in an EMR cluster:
./bin/flink run -m yarn-cluster -yn 25 -yjm 1024 -ytm 4096 -c <class name>
<path to job jar> -planner flink -inputdir xxx
Everything seems to be starting up fine, up to:
All TaskManagers are connected
Using the parallelism provided by the remote cluster (25). To use another
parallelism, set it at the ./bin/flink client.
My main class then tries to parse the passed arguments, and fails.
The arguments being passed to my main() class aren’t what I expect. I get...
-p
lanner
flink
inputdir
s3n://su-wikidump/wikidump-20151112/data/
<s3n://su-wikidump/wikidump-20151112/data/>
It looks like Flink is trying to process all of my arguments, so -planner looks
like “-p”, and -inputdir gets the ‘-‘ consumed before it realizes that there is
no ‘-i<anything>’ parameter that it knows about.
I was assuming that everything after the jar parameter would be sent through as
is.
Any ideas?
Thanks,
— Ken
PS - this is with flink 1.0.2
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com <http://www.scaleunlimited.com/>
custom big data solutions & training
Hadoop, Cascading, Cassandra & Solr
